-- Logs begin at Fri 2022-08-05 18:46:28 UTC, end at Fri 2022-08-05 20:09:47 UTC. -- Aug 05 19:35:22.201777 dv-cinder-852209-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 05 19:35:22.211477 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 05 19:35:22.213511 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 5 19:35:22 2022] *** Aug 05 19:35:22.213511 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 05 19:35:22.213511 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 05 19:35:22.213511 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: nodename: dv-cinder-852209-2-iscsi Aug 05 19:35:22.213511 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: machine: x86_64 Aug 05 19:35:22.213511 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: clock source: unix Aug 05 19:35:22.213511 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: pcre jit disabled Aug 05 19:35:22.213511 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: detected number of CPU cores: 4 Aug 05 19:35:22.213511 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: current working directory: / Aug 05 19:35:22.213511 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: detected binary path: /usr/bin/uwsgi-core Aug 05 19:35:22.214174 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: your processes number limit is 47680 Aug 05 19:35:22.214174 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: your memory page size is 4096 bytes Aug 05 19:35:22.214174 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: detected max file descriptor number: 2048 Aug 05 19:35:22.214174 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: lock engine: pthread robust mutexes Aug 05 19:35:22.214174 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: thunder lock: enabled Aug 05 19:35:22.214174 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 05 19:35:22.214174 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 05 19:35:22.253300 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: Python main interpreter initialized at 0x5566ccc189c0 Aug 05 19:35:22.253300 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: python threads support enabled Aug 05 19:35:22.253300 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: your server socket listen backlog is limited to 100 connections Aug 05 19:35:22.253300 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: your mercy for graceful operations on workers is 90 seconds Aug 05 19:35:22.253704 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: mapped 403077 bytes (393 KB) for 2 cores Aug 05 19:35:22.253780 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: *** Operational MODE: preforking *** Aug 05 19:35:22.254004 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: *** uWSGI is running in multiple interpreter mode *** Aug 05 19:35:22.254004 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: spawned uWSGI master process (pid: 86992) Aug 05 19:35:22.254217 dv-cinder-852209-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 05 19:35:22.255771 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: spawned uWSGI worker 1 (pid: 87004, cores: 1) Aug 05 19:35:22.255771 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: spawned uWSGI worker 2 (pid: 87005, cores: 1) Aug 05 19:35:22.255771 dv-cinder-852209-2-iscsi devstack@keystone.service[86992]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 05 19:35:27.942818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 19:35:27.942818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: from cryptography.utils import int_from_bytes Aug 05 19:35:27.943613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 19:35:27.943613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: from cryptography.utils import int_from_bytes Aug 05 19:35:27.995465 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 19:35:27.995649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 19:35:27.995837 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] command line args: [] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 19:35:27.996188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 19:35:27.996374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ================================================================================ {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 19:35:27.996633 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] admin_token = **** {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.996882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] config_dir = [] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.997072 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.997257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] config_source = [] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.997439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] debug = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.997640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.998021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.998205 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.998460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.999506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.999506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] list_limit = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.999506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] log_config_append = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.999506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.999506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] log_dir = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.999796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] log_file = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:27.999864 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.000110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.000361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.000609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.000789 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.000968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.001215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.001464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.001702 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.001938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.002192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.002408 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.002680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.002931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.003207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.003706 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.86/identity {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.003896 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] publish_errors = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.004136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.004369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.004549 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.004798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.005035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.005272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] standard_threads = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.005505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.005750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.005929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.006106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] use_journal = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.006283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] use_json = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.006458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] use_stderr = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.006674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] use_syslog = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.007070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.007379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.007667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.007933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.008178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.008433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.008676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.008927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.009166 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.009419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.009669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.009919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.010156 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.010407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.010669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.010922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.011172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.011412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.011744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.012014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.012262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.012501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.012760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.013025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.013279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.013531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.013767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.014001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.014239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.014474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.014753 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.015007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.015243 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.015479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.015727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.015978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.016218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.016469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.016704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.016943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.017184 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.017419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.017656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.017892 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.018127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.018368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.018639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.018878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.019118 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.019354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.019594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.019833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.020086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.020322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.020562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.020803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.021043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.021285 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.021524 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.021760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.022013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.022252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.023103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.023389 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.023657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.023899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.024137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.024381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.024629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.024900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.025143 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.025396 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.025646 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.025886 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.026125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.026365 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.026618 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.026883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.027125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.027460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.027702 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.027944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.028184 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.028426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.028665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.028933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.029177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.029425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.029668 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.029914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.030155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.030395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.030648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.030893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.031131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.031368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.031608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.031842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.032086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.032342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.032598 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.032862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.033095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.033361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.033609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.033846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] auth.external = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.034082 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.034382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.034653 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.034888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] auth.password = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.035124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] auth.token = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.035369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.035609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.035861 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.036097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.036348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.036597 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.036833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.037070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] credential.caching = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.037321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.037573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.037826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.038610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.038987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.039530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 19:35:28.039530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: from cryptography.utils import int_from_bytes Aug 05 19:35:28.040053 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.040139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 19:35:28.040139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: from cryptography.utils import int_from_bytes Aug 05 19:35:28.040267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.040267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.040267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.040267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.040512 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.040706 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.040968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.041217 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.041477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.041716 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] federation.caching = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.041955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.042205 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.042452 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.042712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.042965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.043206 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.043462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.043703 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.043962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.044202 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.044453 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.044696 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.caching = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.044949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.045201 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.045438 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.045678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.045928 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.046164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.046401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.046673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.046927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.047185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.047401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.047630 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.047887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.048113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.048356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.048613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.048867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.049141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.049384 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.049625 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.049860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.050100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.050333 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.050570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.050843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.051087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.051351 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.051574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.051824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.052074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.052312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.052562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.052810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.053045 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.053286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.053550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.053988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.054241 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.054937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.054937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.055372 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.055372 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.055570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.055775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.056262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.056262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.056680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.056758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.056947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.057184 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.057419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.057664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.057928 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.058165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.058416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.058691 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.058944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.059183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.059417 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.059655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.059894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.060135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.060369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.060620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.060870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.061119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.061496 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.061749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.061986 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.062238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.062754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.062998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.063239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.063480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.063730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.063982 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.064222 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.064486 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.064720 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.064968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.065207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.065445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.065687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.065937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.066183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.066419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.066679 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.066925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] resource.caching = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.067180 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.067430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.067669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.067922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.068174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.068411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.068661 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.068900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.069170 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.069594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] role.caching = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.069846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] role.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.070084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.070340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.070594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.070872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.071125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.071377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.071628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.071879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.072131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.072368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.072619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.072870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.073120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.073370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.073808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.074052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.074304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.074563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.074844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.075096 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.075336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.075593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.075849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.076092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.076327 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.076563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.076800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.077052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.077395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.077563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.077803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.078042 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.078278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.078516 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] token.caching = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.078782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.079034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.079272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.079531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.079783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.080025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.080272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.080518 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.080768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.081007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.081247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.081483 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.081731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.081979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.082211 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.082454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.082784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.083048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.connection = **** {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.083298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.083548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.083789 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.084036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.084274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.084520 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.085256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.085545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.085847 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.086091 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.086339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.086466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 19:35:28.086579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.086725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 19:35:28.086857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.086939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] command line args: [] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 19:35:28.087099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.087183 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 19:35:28.087302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ================================================================================ {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 19:35:28.087381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.087559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] admin_token = **** {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.087646 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.087804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.087886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] config_dir = [] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.087990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.088069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.088174 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] config_source = [] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.088254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87004) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 19:35:28.088357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] debug = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.088561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.088800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.088980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.089241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.089495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.089725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] list_limit = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.089904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] log_config_append = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.090082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.090260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] log_dir = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.090436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] log_file = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.090693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.090941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.091186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.091430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.091605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.091780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.092019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.092263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.092495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.092728 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.092959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.093189 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.093434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.093678 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.093949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.094444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.86/identity {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.094658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] publish_errors = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.094883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.095113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.095291 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.095535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.095768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.095999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] standard_threads = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.096229 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.096470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.096645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.096820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] use_journal = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.096994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] use_json = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.097167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] use_stderr = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.097340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] use_syslog = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.097568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:35:28.097821 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.098057 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.098316 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.098553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.098823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.099061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.099309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.099550 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.099798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.100044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.100290 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.101182 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.101435 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.101818 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.102073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.102324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.102562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.102848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.103104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.103351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.103586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.103840 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.104102 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.104355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.104604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.104836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.105067 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.105296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.105530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.105779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.106027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.106259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.106489 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.106757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.107005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.107243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.107488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.107721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.107956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.108191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.108423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.108655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.108886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.109117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.109352 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.109606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.109837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.110072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.110303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.110537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.110802 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.111052 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.111284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.111520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.111756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.111992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.112226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.112462 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.112693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.112945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.113182 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.113422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.113655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.113916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.114152 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.114385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.114655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.114880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.115148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.115390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.115639 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.115885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.116153 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.116454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.116791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.117042 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.117283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.117522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.117649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.117793 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.117924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.118056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.118186 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.118319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.118456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.118691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.118929 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.119192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.119434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.119685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.119925 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.120169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.120406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.120645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.120882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.121115 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.121349 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.121581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.121816 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.122049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.122292 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.122529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.122806 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.123046 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.123302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.123572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.123816 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.124051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] auth.external = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.124285 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.124582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.124817 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.125051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] auth.password = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.125283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] auth.token = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.125522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.125756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.126002 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.126233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.126367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.126493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.126736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.126972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.127208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] credential.caching = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.127455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.127705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.127956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.128194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.128439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.128675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.128941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.129196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.129433 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.129691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.129974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.130222 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.130483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.130756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.131014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.131248 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] federation.caching = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.131485 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.131773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.132016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.132251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.132505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.132824 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.133246 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.133670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.133968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.134290 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.134565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.134913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.caching = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.135078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.135323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.135563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.135805 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.136055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.136291 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.136532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.136784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.137038 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.137275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.137519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.137756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.138003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.138253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.138504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.138785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.139036 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.139309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.139555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.139795 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.140029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.140268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.140501 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.140738 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.140983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.141225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.141474 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.141706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.141954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.142202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.142438 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.142705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.142954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.143188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.143430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.143662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.143900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.144138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.144383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.144623 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.144863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.145114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.145364 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.145596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.145829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.146079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.146328 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.146565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.146825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.147063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.147800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.148059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.148327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.148671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.148811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.149059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.149305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.149547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.149779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.150015 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.150249 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.150486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.150739 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.150990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.151237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.151483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.151732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.151978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.152212 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.152463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.152699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.152937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.153173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.153409 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.153653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.153901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.154139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.154392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.154653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.154886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.155123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.155360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.155597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.155983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.156231 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.156467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.156705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.156948 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] resource.caching = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.157027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.157200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.157283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.157449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.157687 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.157937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.158195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.158435 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.158709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.158949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.159189 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.159428 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] role.caching = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.159677 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] role.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.159908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.160158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.160406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.160654 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.160902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.161148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.161395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.161643 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.161891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.162125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.162371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.162618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.162888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.163804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.164098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.164338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.164590 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.164839 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.165087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.165336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.165572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.165827 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.166081 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.166321 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.166582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.166813 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.167043 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.167280 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.167534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.167781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.168099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.168253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] token.caching = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.171848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.173560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.173560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.173560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.173560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.173560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.connection = **** {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.173560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.173560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.174005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.174005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.174136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.174326 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.174525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.174771 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.175018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.175256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.175502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.178492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.178492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.178492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.178492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.178492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.178492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.178492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:35:28.178492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87005) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 19:35:28.202598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.202958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.203050 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.211555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.212137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.233186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.233399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.265704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.265961 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.276438 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.314527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.314849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.335549 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.336311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.336311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.336570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.336570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.336925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.358268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.358503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.358723 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.358945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.359163 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.359380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.360581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.360821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.361051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.361273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.361490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.361715 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.361928 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.373699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.374823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.375053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.375275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.375494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.375711 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.375927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.376139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.384611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.387580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.387828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.388035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.388257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.388475 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.388690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.388906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87004) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.396264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.396443 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.396709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.396935 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.397157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.397476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.398194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87005) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:35:28.406690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.406920 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 19:35:28.407237 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.407444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 19:35:28.407711 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.408020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.408296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.408484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.408673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.408873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 19:35:28.409085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.409246 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 19:35:28.409400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.409672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.409933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.410190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.420311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.420590 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.420862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.421153 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.426686 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.427136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.427562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.428008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.428227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.433264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.433537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.433819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.434102 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.434386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.434691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.434963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.435260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.435551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.435823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.436089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.441339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.449306 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.449584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.449872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.450157 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.450445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.450740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.451015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.451290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.451562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.451837 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.452106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.460808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.462323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.462605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.467353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.467626 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.467904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.474551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.474980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.476346 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.477891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.478176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.479262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.479548 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.479831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.480120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.480416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.480705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.480973 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.481244 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.482882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.483171 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.483455 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.490090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.490374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.495338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.495630 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.495919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.496295 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.496593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.496883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.497163 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.497269 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.497470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.497555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.501422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.504602 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.504898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.509830 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.513029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.514221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.514513 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.516221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.516536 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.516802 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.517095 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.517374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.517654 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.518471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.521649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.521943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.527149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.530617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.533878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.534185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.534349 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.534482 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.534670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.534802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.534951 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.535109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.535223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.535429 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.542193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.543767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.544044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.546890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.547180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.547482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.547771 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.548049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.551862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.552163 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.552424 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.552701 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.559778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.560017 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.560120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.560328 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.560610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.560886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.561169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.561446 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.561570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.561850 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.564970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.565251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.565551 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.565841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.566125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.578401 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.578536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.578792 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.579078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.579359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.579639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.579919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.580208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.581400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.584373 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.588122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.589625 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.589956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.590254 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.596691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.596779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.598294 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.599772 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.601968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.602253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.602538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.602842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.603305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.606069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.607620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.607918 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.608216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.610909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.611196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.611477 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.614697 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.616886 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.619019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87005) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.619301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.619584 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.619859 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.620128 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.620259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.620396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.620552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.620664 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.620838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.620941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.621123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.621335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.621605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.621877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.622139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.622405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.622695 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87005) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.629285 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.629580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.629866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.637055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87004) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:35:28.637334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.637620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.637904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.638329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.638545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.638856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.639148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.639541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.639821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.640100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.640385 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.640660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.640932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87004) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:35:28.656976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5566ccc189c0 pid: 87005 (default app) Aug 05 19:35:28.660582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfdfd2-cfef-47d4-b2d4-9daf3abc0ffb None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:28.660872 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfdfd2-cfef-47d4-b2d4-9daf3abc0ffb None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:28.661145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfdfd2-cfef-47d4-b2d4-9daf3abc0ffb None None] PATH_INFO: `/v3` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:28.665986 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1/1] 10.222.0.86 () {50 vars in 791 bytes} [Fri Aug 5 19:35:28 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:28.675608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5566ccc189c0 pid: 87004 (default app) Aug 05 19:35:36.559119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0d8f6-861a-49f1-918b-e63208a2f9dc None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:36.559681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0d8f6-861a-49f1-918b-e63208a2f9dc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:36.559681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0d8f6-861a-49f1-918b-e63208a2f9dc None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:36.560980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 2/2] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:36.568776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4441887-6c51-4745-b1b4-fa2d381950c6 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:36.569958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4441887-6c51-4745-b1b4-fa2d381950c6 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:36.569958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4441887-6c51-4745-b1b4-fa2d381950c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:36.641377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: INFO dbcounter [None req-a4441887-6c51-4745-b1b4-fa2d381950c6 None None] Registered counter for database keystone Aug 05 19:35:36.660807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG oslo_db.sqlalchemy.engines [None req-a4441887-6c51-4745-b1b4-fa2d381950c6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87004) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 19:35:36.667711 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writer thread running {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 19:35:36.864102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG passlib.handlers.bcrypt [None req-a4441887-6c51-4745-b1b4-fa2d381950c6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87004) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 19:35:36.864527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG passlib.handlers.bcrypt [None req-a4441887-6c51-4745-b1b4-fa2d381950c6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87004) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 19:35:36.921981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-a4441887-6c51-4745-b1b4-fa2d381950c6 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:36.923657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a4441887-6c51-4745-b1b4-fa2d381950c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:37.062845 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1/3] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 05 19:35:37.207899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0d8f6-861a-49f1-918b-e63208a2f9dc None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:37.287130 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: INFO dbcounter [None req-edf0d8f6-861a-49f1-918b-e63208a2f9dc None None] Registered counter for database keystone Aug 05 19:35:37.309265 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG oslo_db.sqlalchemy.engines [None req-edf0d8f6-861a-49f1-918b-e63208a2f9dc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87005) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 19:35:37.315720 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writer thread running {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 19:35:37.434125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0d8f6-861a-49f1-918b-e63208a2f9dc None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:37.438716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3c2f8-9fd2-4880-8863-43880a2f1f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:37.439372 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3c2f8-9fd2-4880-8863-43880a2f1f9d admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:37.439689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3c2f8-9fd2-4880-8863-43880a2f1f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:37.439994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3c2f8-9fd2-4880-8863-43880a2f1f9d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:37.448826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3c2f8-9fd2-4880-8863-43880a2f1f9d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:37.500659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fbe3c2f8-9fd2-4880-8863-43880a2f1f9d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 19:35:37.501007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3c2f8-9fd2-4880-8863-43880a2f1f9d admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:37.512778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 3/4] 10.222.0.86 () {58 vars in 1324 bytes} [Fri Aug 5 19:35:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 19:35:37.518846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4441887-6c51-4745-b1b4-fa2d381950c6 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:37.534454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4441887-6c51-4745-b1b4-fa2d381950c6 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:37.539906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795feb3-7a8a-41d7-a3ea-ca6692afcfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:37.540861 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795feb3-7a8a-41d7-a3ea-ca6692afcfd3 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:37.540861 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795feb3-7a8a-41d7-a3ea-ca6692afcfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:37.541081 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795feb3-7a8a-41d7-a3ea-ca6692afcfd3 admin admin] PATH_INFO: `/v3/projects/11aa9c879322427d8920955a2aa93539` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:37.542431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795feb3-7a8a-41d7-a3ea-ca6692afcfd3 admin admin] RBAC: Authorizing `identity:get_project(project_id=11aa9c879322427d8920955a2aa93539)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:37.593709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e795feb3-7a8a-41d7-a3ea-ca6692afcfd3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 19:35:37.594292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795feb3-7a8a-41d7-a3ea-ca6692afcfd3 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:37.598700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 2/5] 10.222.0.86 () {56 vars in 1207 bytes} [Fri Aug 5 19:35:37 2022] GET /identity/v3/projects/11aa9c879322427d8920955a2aa93539 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:39.627048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb5a13-c378-4010-9071-22f2d1ebac23 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:39.627595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb5a13-c378-4010-9071-22f2d1ebac23 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:39.627595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb5a13-c378-4010-9071-22f2d1ebac23 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:39.632032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 4/6] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:39.637723 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7228ec1-5356-4c56-89f5-c6def5fd32bc None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:39.638199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7228ec1-5356-4c56-89f5-c6def5fd32bc None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:39.638288 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7228ec1-5356-4c56-89f5-c6def5fd32bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:39.659272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-a7228ec1-5356-4c56-89f5-c6def5fd32bc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:39.660779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a7228ec1-5356-4c56-89f5-c6def5fd32bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:39.670051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 3/7] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 05 19:35:39.694779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb5a13-c378-4010-9071-22f2d1ebac23 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:39.707160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb5a13-c378-4010-9071-22f2d1ebac23 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:39.712002 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cfe69-40f7-4ae1-987f-2ecd2a6efae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:39.712761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cfe69-40f7-4ae1-987f-2ecd2a6efae8 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:39.713045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cfe69-40f7-4ae1-987f-2ecd2a6efae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:39.713315 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cfe69-40f7-4ae1-987f-2ecd2a6efae8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:39.721395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cfe69-40f7-4ae1-987f-2ecd2a6efae8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:39.723211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cfe69-40f7-4ae1-987f-2ecd2a6efae8 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:39.733104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 5/8] 10.222.0.86 () {58 vars in 1324 bytes} [Fri Aug 5 19:35:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:35:39.738751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7228ec1-5356-4c56-89f5-c6def5fd32bc None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:39.748596 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7228ec1-5356-4c56-89f5-c6def5fd32bc None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:39.753386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d53b13-2ccd-45e0-aa48-94f472d0f69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:39.754071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d53b13-2ccd-45e0-aa48-94f472d0f69d admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:39.754375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d53b13-2ccd-45e0-aa48-94f472d0f69d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:39.754705 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d53b13-2ccd-45e0-aa48-94f472d0f69d admin admin] PATH_INFO: `/v3/users/047b4b41672048d8889a2b3bb55b2d97` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:39.756146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d53b13-2ccd-45e0-aa48-94f472d0f69d admin admin] RBAC: Authorizing `identity:get_user(user_id=047b4b41672048d8889a2b3bb55b2d97)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:39.758870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d53b13-2ccd-45e0-aa48-94f472d0f69d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:39.760560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 4/9] 10.222.0.86 () {56 vars in 1198 bytes} [Fri Aug 5 19:35:39 2022] GET /identity/v3/users/047b4b41672048d8889a2b3bb55b2d97 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:41.849906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8698ff-7b3a-45bf-b23b-a2429101d559 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:41.850450 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8698ff-7b3a-45bf-b23b-a2429101d559 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:41.850450 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8698ff-7b3a-45bf-b23b-a2429101d559 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:41.852009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 6/10] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:41.857464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c83a56-7f78-4dff-9202-66a40123606d None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:41.857895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c83a56-7f78-4dff-9202-66a40123606d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:41.858098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c83a56-7f78-4dff-9202-66a40123606d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:41.877777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-12c83a56-7f78-4dff-9202-66a40123606d None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:41.879050 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-12c83a56-7f78-4dff-9202-66a40123606d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:41.961882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a45ba-b2e6-4009-beac-99942c0cccdc None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:41.962148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a45ba-b2e6-4009-beac-99942c0cccdc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:41.962424 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a45ba-b2e6-4009-beac-99942c0cccdc None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:41.963812 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 7/11] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:35:41.966975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 5/12] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:35:41.969476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cff26a-57ff-4ea9-9554-64bc715a01d0 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:41.969755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cff26a-57ff-4ea9-9554-64bc715a01d0 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:41.970026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cff26a-57ff-4ea9-9554-64bc715a01d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:41.989810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c83a56-7f78-4dff-9202-66a40123606d None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:42.013553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG passlib.handlers.bcrypt [None req-c9cff26a-57ff-4ea9-9554-64bc715a01d0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87005) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 19:35:42.013982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG passlib.handlers.bcrypt [None req-c9cff26a-57ff-4ea9-9554-64bc715a01d0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87005) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 19:35:42.016771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:42.060266 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c9cff26a-57ff-4ea9-9554-64bc715a01d0 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:42.061900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c9cff26a-57ff-4ea9-9554-64bc715a01d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:42.104809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c83a56-7f78-4dff-9202-66a40123606d None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:42.140326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf68f7-7e5d-47c1-860e-7de1173bba07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:42.140976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf68f7-7e5d-47c1-860e-7de1173bba07 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:42.141261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf68f7-7e5d-47c1-860e-7de1173bba07 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:42.141536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf68f7-7e5d-47c1-860e-7de1173bba07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:42.154019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf68f7-7e5d-47c1-860e-7de1173bba07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-29cf68f7-7e5d-47c1-860e-7de1173bba07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:35:42.163246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:35:42.164330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:35:42.165190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:35:42.165190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:35:42.165190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:35:42.165190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:35:42.165190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:35:42.165190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:35:42.165190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:35:42.165190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:35:42.165190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:35:42.165190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:35:42.165190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:35:42.165190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:35:42.165190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 6/13] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:35:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:35:42.168547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 8/14] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:35:42.171098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf68f7-7e5d-47c1-860e-7de1173bba07 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:42.190574 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cff26a-57ff-4ea9-9554-64bc715a01d0 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:42.265033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:42.284223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf68f7-7e5d-47c1-860e-7de1173bba07 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:42.298645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cff26a-57ff-4ea9-9554-64bc715a01d0 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:42.301119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:42.334074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d915d40-73e7-4b11-9410-6b6c4f6a3e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:42.334733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915d40-73e7-4b11-9410-6b6c4f6a3e74 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:42.335023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915d40-73e7-4b11-9410-6b6c4f6a3e74 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:42.335303 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915d40-73e7-4b11-9410-6b6c4f6a3e74 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:42.336039 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d915d40-73e7-4b11-9410-6b6c4f6a3e74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:35:42.336347 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d915d40-73e7-4b11-9410-6b6c4f6a3e74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:42.338748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d915d40-73e7-4b11-9410-6b6c4f6a3e74 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:42.342957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b794b3-6386-4708-8f23-8d9e0dc2b843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:42.343832 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b794b3-6386-4708-8f23-8d9e0dc2b843 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:42.344113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b794b3-6386-4708-8f23-8d9e0dc2b843 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:42.344383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b794b3-6386-4708-8f23-8d9e0dc2b843 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:42.352827 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 7/15] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:35:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:42.358707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d915d40-73e7-4b11-9410-6b6c4f6a3e74 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:42.417253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b794b3-6386-4708-8f23-8d9e0dc2b843 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:42.418867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b794b3-6386-4708-8f23-8d9e0dc2b843 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:42.532611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 9/16] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:35:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:35:42.538275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b794b3-6386-4708-8f23-8d9e0dc2b843 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:42.542362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:42.583722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d915d40-73e7-4b11-9410-6b6c4f6a3e74 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:42.626212 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be1c19-ffe0-4b3d-9308-2a7922033530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:42.627175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be1c19-ffe0-4b3d-9308-2a7922033530 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:42.627642 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be1c19-ffe0-4b3d-9308-2a7922033530 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:42.628094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be1c19-ffe0-4b3d-9308-2a7922033530 None admin] PATH_INFO: `/v3/users/047b4b41672048d8889a2b3bb55b2d97` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:42.631629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be1c19-ffe0-4b3d-9308-2a7922033530 None admin] RBAC: Authorizing `identity:get_user(user_id=047b4b41672048d8889a2b3bb55b2d97)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:42.634812 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be1c19-ffe0-4b3d-9308-2a7922033530 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:42.643519 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 8/17] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:35:42 2022] GET /identity/v3/users/047b4b41672048d8889a2b3bb55b2d97 => generated 247 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:35:42.644621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be1c19-ffe0-4b3d-9308-2a7922033530 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:42.649724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b794b3-6386-4708-8f23-8d9e0dc2b843 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:42.659871 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:42.686395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc903e-988f-4816-b1e6-e9093dc6af0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:42.687363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc903e-988f-4816-b1e6-e9093dc6af0b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:42.687896 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc903e-988f-4816-b1e6-e9093dc6af0b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:42.688167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc903e-988f-4816-b1e6-e9093dc6af0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:42.707525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc903e-988f-4816-b1e6-e9093dc6af0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:42.709006 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc903e-988f-4816-b1e6-e9093dc6af0b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-a3cc903e-988f-4816-b1e6-e9093dc6af0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:35:42.732207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:35:42.733150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:35:42.733991 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:35:42.733991 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:35:42.733991 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:35:42.733991 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:35:42.733991 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 10/18] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:35:42.739307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc903e-988f-4816-b1e6-e9093dc6af0b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:42.754669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be1c19-ffe0-4b3d-9308-2a7922033530 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:42.790023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9aefe5-3d13-4670-a793-3a825acae9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:42.791113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9aefe5-3d13-4670-a793-3a825acae9b2 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:42.791454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9aefe5-3d13-4670-a793-3a825acae9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:42.791729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9aefe5-3d13-4670-a793-3a825acae9b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:42.793039 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9aefe5-3d13-4670-a793-3a825acae9b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:42.795086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9aefe5-3d13-4670-a793-3a825acae9b2 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:42.796271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 9/19] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:35:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:35:42.798913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:42.802736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9aefe5-3d13-4670-a793-3a825acae9b2 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:42.853684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc903e-988f-4816-b1e6-e9093dc6af0b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:42.891918 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:42.895377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be299b-69f7-4f56-a886-80305dce454e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:42.896008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be299b-69f7-4f56-a886-80305dce454e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:42.896286 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be299b-69f7-4f56-a886-80305dce454e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:42.896556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be299b-69f7-4f56-a886-80305dce454e None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:42.897268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be299b-69f7-4f56-a886-80305dce454e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:35:42.897569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be299b-69f7-4f56-a886-80305dce454e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:42.899002 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be299b-69f7-4f56-a886-80305dce454e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:42.912968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9aefe5-3d13-4670-a793-3a825acae9b2 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:42.919420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 11/20] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:35:42.966302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12876426-db21-457b-97ef-f8bece0721b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:42.967086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12876426-db21-457b-97ef-f8bece0721b8 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:42.967488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12876426-db21-457b-97ef-f8bece0721b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:42.967763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12876426-db21-457b-97ef-f8bece0721b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:42.968595 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12876426-db21-457b-97ef-f8bece0721b8 None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.domain.id=default, user.id=047b4b41672048d8889a2b3bb55b2d97) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:35:42.968904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12876426-db21-457b-97ef-f8bece0721b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:42.970434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12876426-db21-457b-97ef-f8bece0721b8 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:42.979349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 10/21] 10.222.0.86 () {56 vars in 1322 bytes} [Fri Aug 5 19:35:42 2022] GET /identity/v3/role_assignments?user.id=047b4b41672048d8889a2b3bb55b2d97&scope.domain.id=default&role.id=93eb204498fc49d297bcd1841e00818d => generated 233 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:35:45.500744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263f45d-28f6-4dd4-ad89-afe19ee5e300 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:45.501249 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263f45d-28f6-4dd4-ad89-afe19ee5e300 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:45.501353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263f45d-28f6-4dd4-ad89-afe19ee5e300 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:45.502977 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 12/22] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:45.512273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f8b84-a289-41e3-984f-a3389be345ee None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:45.513504 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f8b84-a289-41e3-984f-a3389be345ee None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:45.514032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f8b84-a289-41e3-984f-a3389be345ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:45.544679 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-326f8b84-a289-41e3-984f-a3389be345ee None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:45.546238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-326f8b84-a289-41e3-984f-a3389be345ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:45.650194 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 11/23] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:35:45.690528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263f45d-28f6-4dd4-ad89-afe19ee5e300 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:45.726744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53864f2f-bf78-473a-9210-3a27bb94b1c1 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:45.727416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53864f2f-bf78-473a-9210-3a27bb94b1c1 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:45.727903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53864f2f-bf78-473a-9210-3a27bb94b1c1 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:45.729464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 12/24] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:35:45.731402 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93b234-a1e6-4275-97ee-fbbc2cacb182 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:45.731910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93b234-a1e6-4275-97ee-fbbc2cacb182 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:45.732400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93b234-a1e6-4275-97ee-fbbc2cacb182 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:45.734103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 13/25] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:45.740150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df12fd0-b59e-4c5b-8d12-f8bdd88fc2cb None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:45.740861 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df12fd0-b59e-4c5b-8d12-f8bdd88fc2cb None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:45.741332 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df12fd0-b59e-4c5b-8d12-f8bdd88fc2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:45.771462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-6df12fd0-b59e-4c5b-8d12-f8bdd88fc2cb None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:45.774025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6df12fd0-b59e-4c5b-8d12-f8bdd88fc2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:45.895262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 14/26] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:35:45.897139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12faf6-1e62-4453-96c7-b2d02c50322a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:45.897639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12faf6-1e62-4453-96c7-b2d02c50322a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:45.898243 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12faf6-1e62-4453-96c7-b2d02c50322a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:45.912680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:45.921070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263f45d-28f6-4dd4-ad89-afe19ee5e300 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:45.936435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-8b12faf6-1e62-4453-96c7-b2d02c50322a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:45.937880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8b12faf6-1e62-4453-96c7-b2d02c50322a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:45.958993 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:45.997288 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c86c3c-4821-4416-bc80-dba9a567cb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:45.997866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c86c3c-4821-4416-bc80-dba9a567cb87 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:45.998145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c86c3c-4821-4416-bc80-dba9a567cb87 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:45.998417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c86c3c-4821-4416-bc80-dba9a567cb87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:46.032293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c86c3c-4821-4416-bc80-dba9a567cb87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:35:46.054533 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 15/27] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-65c86c3c-4821-4416-bc80-dba9a567cb87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:35:46.055384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:35:46.056449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:35:46.057220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:35:46.057220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:35:46.057220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:35:46.057220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:35:46.057220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:35:46.057220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:35:46.057220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:35:46.057220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:35:46.057220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:35:46.057220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:35:46.057220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:35:46.057220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:35:46.057220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 13/28] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:35:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:35:46.064332 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12faf6-1e62-4453-96c7-b2d02c50322a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:46.072397 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c86c3c-4821-4416-bc80-dba9a567cb87 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:46.215501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12faf6-1e62-4453-96c7-b2d02c50322a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:46.226875 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c86c3c-4821-4416-bc80-dba9a567cb87 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:46.257838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf3772-da3f-4771-bb02-5eb1429fc8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:46.258370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf3772-da3f-4771-bb02-5eb1429fc8a6 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:46.259021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf3772-da3f-4771-bb02-5eb1429fc8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:46.259953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf3772-da3f-4771-bb02-5eb1429fc8a6 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:46.260798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf3772-da3f-4771-bb02-5eb1429fc8a6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:46.262524 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf3772-da3f-4771-bb02-5eb1429fc8a6 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:46.266778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:46.276257 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fba30-0f80-4d1d-8ccc-2a5356501478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:46.277693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fba30-0f80-4d1d-8ccc-2a5356501478 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:46.278467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fba30-0f80-4d1d-8ccc-2a5356501478 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:46.279137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fba30-0f80-4d1d-8ccc-2a5356501478 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:46.280088 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fba30-0f80-4d1d-8ccc-2a5356501478 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:35:46.280630 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fba30-0f80-4d1d-8ccc-2a5356501478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:46.282239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fba30-0f80-4d1d-8ccc-2a5356501478 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:46.299601 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 14/29] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:35:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:46.301178 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77858a99-626f-4089-8204-d4bd2b5947bc None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:46.301655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77858a99-626f-4089-8204-d4bd2b5947bc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:46.302170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77858a99-626f-4089-8204-d4bd2b5947bc None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:46.303628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 15/30] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:46.304830 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77858a99-626f-4089-8204-d4bd2b5947bc None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:46.399609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 16/31] 10.222.0.86 () {60 vars in 1131 bytes} [Fri Aug 5 19:35:46 2022] POST /identity/v3/roles => generated 225 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:35:46.401188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf3772-da3f-4771-bb02-5eb1429fc8a6 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:46.538430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:46.612046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf3772-da3f-4771-bb02-5eb1429fc8a6 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:46.648913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77858a99-626f-4089-8204-d4bd2b5947bc None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:46.668210 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc33153-85ad-4728-a45e-62efd0b6c9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:46.669121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc33153-85ad-4728-a45e-62efd0b6c9ca None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:46.669852 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc33153-85ad-4728-a45e-62efd0b6c9ca None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:46.670274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc33153-85ad-4728-a45e-62efd0b6c9ca None admin] PATH_INFO: `/v3/users/047b4b41672048d8889a2b3bb55b2d97` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:46.675167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc33153-85ad-4728-a45e-62efd0b6c9ca None admin] RBAC: Authorizing `identity:get_user(user_id=047b4b41672048d8889a2b3bb55b2d97)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:46.677008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc33153-85ad-4728-a45e-62efd0b6c9ca None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:46.679261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 17/32] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:35:46 2022] GET /identity/v3/users/047b4b41672048d8889a2b3bb55b2d97 => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:46.682126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa27263-6c06-46d1-8a5c-3d9839af64ae None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:46.682665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa27263-6c06-46d1-8a5c-3d9839af64ae None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:46.683122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa27263-6c06-46d1-8a5c-3d9839af64ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:46.696159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a52973-019f-4c10-adc3-85dea80eb37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:46.697241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a52973-019f-4c10-adc3-85dea80eb37c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:46.697683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a52973-019f-4c10-adc3-85dea80eb37c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:46.698778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a52973-019f-4c10-adc3-85dea80eb37c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:46.705006 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-5aa27263-6c06-46d1-8a5c-3d9839af64ae None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:46.706712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5aa27263-6c06-46d1-8a5c-3d9839af64ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:46.713634 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a52973-019f-4c10-adc3-85dea80eb37c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:46.715495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a52973-019f-4c10-adc3-85dea80eb37c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-c3a52973-019f-4c10-adc3-85dea80eb37c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:35:46.738084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:35:46.739751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:35:46.740736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:35:46.740736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:35:46.740736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:35:46.740736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:35:46.743532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 16/33] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 438 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:35:46.744637 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a52973-019f-4c10-adc3-85dea80eb37c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:46.800768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:46.819044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 18/34] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:35:46.823953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa27263-6c06-46d1-8a5c-3d9839af64ae None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:46.876085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:46.885683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a52973-019f-4c10-adc3-85dea80eb37c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:46.928232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62944a4-e3ae-4e29-818d-105458c3e2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:46.928889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62944a4-e3ae-4e29-818d-105458c3e2f3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:46.929189 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62944a4-e3ae-4e29-818d-105458c3e2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:46.929456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62944a4-e3ae-4e29-818d-105458c3e2f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:46.931601 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62944a4-e3ae-4e29-818d-105458c3e2f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:46.933090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62944a4-e3ae-4e29-818d-105458c3e2f3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:46.934953 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 17/35] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:35:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:46.940290 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e6b13-95a7-47fd-8ac4-bd0ac07b1ca9 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:46.940582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e6b13-95a7-47fd-8ac4-bd0ac07b1ca9 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:46.940861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e6b13-95a7-47fd-8ac4-bd0ac07b1ca9 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:46.942741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 18/36] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:46.946698 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e6b13-95a7-47fd-8ac4-bd0ac07b1ca9 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:46.948241 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa27263-6c06-46d1-8a5c-3d9839af64ae None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:46.989068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dcbd4-d592-4123-b2a7-46f0395a0e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:46.989703 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dcbd4-d592-4123-b2a7-46f0395a0e47 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:46.989990 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dcbd4-d592-4123-b2a7-46f0395a0e47 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:46.990269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9dcbd4-d592-4123-b2a7-46f0395a0e47 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:46.991022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dcbd4-d592-4123-b2a7-46f0395a0e47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:35:46.991336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dcbd4-d592-4123-b2a7-46f0395a0e47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:46.992748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9dcbd4-d592-4123-b2a7-46f0395a0e47 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:47.018507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 19/37] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:47.018507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dcbd4-d592-4123-b2a7-46f0395a0e47 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:47.075382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:47.082990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e6b13-95a7-47fd-8ac4-bd0ac07b1ca9 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:47.122926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df973363-0228-4ed1-af04-379185ed26e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:47.123556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df973363-0228-4ed1-af04-379185ed26e2 None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:47.123836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df973363-0228-4ed1-af04-379185ed26e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:47.124109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df973363-0228-4ed1-af04-379185ed26e2 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:47.124829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df973363-0228-4ed1-af04-379185ed26e2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:47.126221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df973363-0228-4ed1-af04-379185ed26e2 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:47.133058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9dcbd4-d592-4123-b2a7-46f0395a0e47 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:47.172066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930aaa2a-7d29-402c-88d2-f2efaa1c6a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:47.172735 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930aaa2a-7d29-402c-88d2-f2efaa1c6a72 None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:47.173015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930aaa2a-7d29-402c-88d2-f2efaa1c6a72 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:47.173290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930aaa2a-7d29-402c-88d2-f2efaa1c6a72 None admin] PATH_INFO: `/v3/domains/default/users/047b4b41672048d8889a2b3bb55b2d97/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:47.175333 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930aaa2a-7d29-402c-88d2-f2efaa1c6a72 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=047b4b41672048d8889a2b3bb55b2d97, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:47.176777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930aaa2a-7d29-402c-88d2-f2efaa1c6a72 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:47.191180 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 20/38] 10.222.0.86 () {58 vars in 1361 bytes} [Fri Aug 5 19:35:47 2022] PUT /identity/v3/domains/default/users/047b4b41672048d8889a2b3bb55b2d97/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:35:47.192493 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aec1bd-f613-4b00-b5cf-98e62bb1c9e0 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:47.192788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aec1bd-f613-4b00-b5cf-98e62bb1c9e0 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:47.193061 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aec1bd-f613-4b00-b5cf-98e62bb1c9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:47.221558 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e0aec1bd-f613-4b00-b5cf-98e62bb1c9e0 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:47.223113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e0aec1bd-f613-4b00-b5cf-98e62bb1c9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:47.250463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 19/39] 10.222.0.86 () {60 vars in 1131 bytes} [Fri Aug 5 19:35:46 2022] POST /identity/v3/roles => generated 219 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:35:47.254683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df973363-0228-4ed1-af04-379185ed26e2 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:47.303066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:47.327725 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 21/40] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:35:47.330297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1773-bda0-4753-84e0-523578822811 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:47.330889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1773-bda0-4753-84e0-523578822811 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:47.331674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1773-bda0-4753-84e0-523578822811 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:47.333631 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 22/41] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:47.348182 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9b86b-3ac2-4697-bc31-6f1c12ac7e0c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:47.348758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9b86b-3ac2-4697-bc31-6f1c12ac7e0c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:47.349200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9b86b-3ac2-4697-bc31-6f1c12ac7e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:47.369971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e4f9b86b-3ac2-4697-bc31-6f1c12ac7e0c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:47.371897 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e4f9b86b-3ac2-4697-bc31-6f1c12ac7e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:47.396584 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df973363-0228-4ed1-af04-379185ed26e2 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:47.434101 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:47.459533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f66c1-7a40-403b-8102-e09aad1762a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:47.459533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f66c1-7a40-403b-8102-e09aad1762a8 None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:47.459533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f66c1-7a40-403b-8102-e09aad1762a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:47.459533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f66c1-7a40-403b-8102-e09aad1762a8 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:47.460881 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f66c1-7a40-403b-8102-e09aad1762a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:47.463353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f66c1-7a40-403b-8102-e09aad1762a8 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:47.473486 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 23/42] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:35:47.474805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9b86b-3ac2-4697-bc31-6f1c12ac7e0c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:47.511950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 20/43] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:35:47 2022] POST /identity/v3/projects => generated 301 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:35:47.512847 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f66c1-7a40-403b-8102-e09aad1762a8 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:47.647120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:47.655577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9b86b-3ac2-4697-bc31-6f1c12ac7e0c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:47.701205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f66c1-7a40-403b-8102-e09aad1762a8 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:47.716523 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a3aff-5e1c-47b0-9264-a80adcec2582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:47.717582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a3aff-5e1c-47b0-9264-a80adcec2582 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:47.718064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a3aff-5e1c-47b0-9264-a80adcec2582 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:47.718582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a3aff-5e1c-47b0-9264-a80adcec2582 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:47.719540 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a3aff-5e1c-47b0-9264-a80adcec2582 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:47.721231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a3aff-5e1c-47b0-9264-a80adcec2582 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:47.740088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 24/44] 10.222.0.86 () {60 vars in 1131 bytes} [Fri Aug 5 19:35:47 2022] POST /identity/v3/roles => generated 223 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:35:47.752961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d1556-2026-4a9b-8cd5-90ddcc9ecc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:47.754408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d1556-2026-4a9b-8cd5-90ddcc9ecc5c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:47.754967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d1556-2026-4a9b-8cd5-90ddcc9ecc5c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:47.755495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d1556-2026-4a9b-8cd5-90ddcc9ecc5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:47.757262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d1556-2026-4a9b-8cd5-90ddcc9ecc5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:47.759813 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d1556-2026-4a9b-8cd5-90ddcc9ecc5c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:47.762498 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 21/45] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:35:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:47.770897 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a3aff-5e1c-47b0-9264-a80adcec2582 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:47.820907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:47.934514 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a3aff-5e1c-47b0-9264-a80adcec2582 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:47.997645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232da90-45ab-45a5-ade2-02d9a4fd66c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:47.998290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232da90-45ab-45a5-ade2-02d9a4fd66c3 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:47.998573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232da90-45ab-45a5-ade2-02d9a4fd66c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:47.998912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232da90-45ab-45a5-ade2-02d9a4fd66c3 None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:47.999697 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232da90-45ab-45a5-ade2-02d9a4fd66c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:48.001196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232da90-45ab-45a5-ade2-02d9a4fd66c3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:48.043062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 25/46] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:35:47 2022] POST /identity/v3/projects => generated 312 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:35:49.402649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f945-6b52-4612-9bdb-b8803b842187 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:49.403173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f945-6b52-4612-9bdb-b8803b842187 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:49.403445 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f945-6b52-4612-9bdb-b8803b842187 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:49.405172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 22/47] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:49.410521 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32cd82-7561-4add-bf1e-f707ce3a521f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:49.411005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32cd82-7561-4add-bf1e-f707ce3a521f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:49.411085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32cd82-7561-4add-bf1e-f707ce3a521f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:49.429994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-2d32cd82-7561-4add-bf1e-f707ce3a521f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:49.431461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2d32cd82-7561-4add-bf1e-f707ce3a521f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:49.509076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 26/48] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:35:49.532762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602f945-6b52-4612-9bdb-b8803b842187 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:49.640558 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602f945-6b52-4612-9bdb-b8803b842187 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:49.675000 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0297484-7db3-42a3-812a-5840dc362687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:49.675564 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0297484-7db3-42a3-812a-5840dc362687 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:49.675845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0297484-7db3-42a3-812a-5840dc362687 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:49.676119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0297484-7db3-42a3-812a-5840dc362687 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:49.683504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:49.688523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0297484-7db3-42a3-812a-5840dc362687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-f0297484-7db3-42a3-812a-5840dc362687 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:35:49.693792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:35:49.694796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:35:49.695628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:35:49.695628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:35:49.695628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:35:49.695628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:35:49.695628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:35:49.695628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:35:49.695628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:35:49.695628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:35:49.695628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:35:49.695628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:35:49.695628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:35:49.695628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:35:49.695628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 23/49] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:35:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:35:49.701579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32cd82-7561-4add-bf1e-f707ce3a521f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:49.717651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:49.817811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32cd82-7561-4add-bf1e-f707ce3a521f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:49.856270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45985a0-4890-4042-962c-b1fdec581449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:49.856894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45985a0-4890-4042-962c-b1fdec581449 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:49.857174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45985a0-4890-4042-962c-b1fdec581449 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:49.857447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45985a0-4890-4042-962c-b1fdec581449 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:49.858167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45985a0-4890-4042-962c-b1fdec581449 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:35:49.858471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45985a0-4890-4042-962c-b1fdec581449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:49.859898 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45985a0-4890-4042-962c-b1fdec581449 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:49.868939 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 27/50] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:35:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:49.874865 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0297484-7db3-42a3-812a-5840dc362687 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:49.985981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0297484-7db3-42a3-812a-5840dc362687 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:50.023280 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9383e3c-d402-4184-b62c-61f3d6c0b18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:50.023925 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9383e3c-d402-4184-b62c-61f3d6c0b18e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:50.024350 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9383e3c-d402-4184-b62c-61f3d6c0b18e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:50.024765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9383e3c-d402-4184-b62c-61f3d6c0b18e None admin] PATH_INFO: `/v3/users/047b4b41672048d8889a2b3bb55b2d97` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:50.026243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9383e3c-d402-4184-b62c-61f3d6c0b18e None admin] RBAC: Authorizing `identity:get_user(user_id=047b4b41672048d8889a2b3bb55b2d97)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:50.027744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9383e3c-d402-4184-b62c-61f3d6c0b18e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:50.029379 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 24/51] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:35:49 2022] GET /identity/v3/users/047b4b41672048d8889a2b3bb55b2d97 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:50.036247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45985a0-4890-4042-962c-b1fdec581449 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:50.119280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:50.141904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9858d-4a9f-4ba2-8f02-48cb06166d0e None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:50.142181 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9858d-4a9f-4ba2-8f02-48cb06166d0e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:50.142457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9858d-4a9f-4ba2-8f02-48cb06166d0e None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:50.143888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 25/52] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:35:50.147661 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45985a0-4890-4042-962c-b1fdec581449 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:50.150053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009de92b-4b72-4119-9396-afbd25ddbfbd None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:50.150053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009de92b-4b72-4119-9396-afbd25ddbfbd None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:50.150646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009de92b-4b72-4119-9396-afbd25ddbfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:50.174902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-009de92b-4b72-4119-9396-afbd25ddbfbd None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:50.177792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-009de92b-4b72-4119-9396-afbd25ddbfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:50.187015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d15b8-2fa5-4598-be92-c84be6355d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:50.187914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d15b8-2fa5-4598-be92-c84be6355d8f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:50.188199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d15b8-2fa5-4598-be92-c84be6355d8f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:50.188471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d15b8-2fa5-4598-be92-c84be6355d8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:50.189645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d15b8-2fa5-4598-be92-c84be6355d8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:50.191136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d15b8-2fa5-4598-be92-c84be6355d8f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:50.192736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 28/53] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:35:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:50.199645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d15b8-2fa5-4598-be92-c84be6355d8f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:50.244072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:50.252873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 26/54] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:35:50.277346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009de92b-4b72-4119-9396-afbd25ddbfbd None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:50.313092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d15b8-2fa5-4598-be92-c84be6355d8f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:50.349067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f67904-faf5-46a9-927a-fc9cf963dd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:50.349694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f67904-faf5-46a9-927a-fc9cf963dd8c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:50.349980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f67904-faf5-46a9-927a-fc9cf963dd8c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:50.350265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f67904-faf5-46a9-927a-fc9cf963dd8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:50.351110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f67904-faf5-46a9-927a-fc9cf963dd8c None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.domain.id=default, user.id=047b4b41672048d8889a2b3bb55b2d97) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:35:50.351425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f67904-faf5-46a9-927a-fc9cf963dd8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:50.353487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f67904-faf5-46a9-927a-fc9cf963dd8c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:50.359955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 29/55] 10.222.0.86 () {56 vars in 1322 bytes} [Fri Aug 5 19:35:50 2022] GET /identity/v3/role_assignments?user.id=047b4b41672048d8889a2b3bb55b2d97&scope.domain.id=default&role.id=93eb204498fc49d297bcd1841e00818d => generated 530 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:35:50.392046 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009de92b-4b72-4119-9396-afbd25ddbfbd None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:50.439373 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43656d4a-5fd3-41ad-af66-eaf18487fca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:50.440096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43656d4a-5fd3-41ad-af66-eaf18487fca4 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:50.440318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43656d4a-5fd3-41ad-af66-eaf18487fca4 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:50.440594 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43656d4a-5fd3-41ad-af66-eaf18487fca4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:50.441923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43656d4a-5fd3-41ad-af66-eaf18487fca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:50.447325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43656d4a-5fd3-41ad-af66-eaf18487fca4 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:50.447325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 27/56] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:35:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:35:50.454588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f67904-faf5-46a9-927a-fc9cf963dd8c None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:50.467937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:50.574978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f67904-faf5-46a9-927a-fc9cf963dd8c None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:50.611888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c1f24-6921-401c-8ffc-4631f331c626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:50.612506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c1f24-6921-401c-8ffc-4631f331c626 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:50.612785 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c1f24-6921-401c-8ffc-4631f331c626 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:50.613055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c1f24-6921-401c-8ffc-4631f331c626 None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:50.613741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c1f24-6921-401c-8ffc-4631f331c626 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:50.616251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c1f24-6921-401c-8ffc-4631f331c626 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:50.643020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 30/57] 10.222.0.86 () {60 vars in 1140 bytes} [Fri Aug 5 19:35:50 2022] POST /identity/v3/projects => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:35:52.696014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6674fd-1b66-43cf-9b10-ce201a8ce5f8 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:52.696533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6674fd-1b66-43cf-9b10-ce201a8ce5f8 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:52.696648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6674fd-1b66-43cf-9b10-ce201a8ce5f8 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:52.697942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 28/58] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:52.703416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df6b5f-a88d-46e7-b0ce-c5052bd4aad5 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:52.703879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df6b5f-a88d-46e7-b0ce-c5052bd4aad5 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:52.703964 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df6b5f-a88d-46e7-b0ce-c5052bd4aad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:52.722294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-60df6b5f-a88d-46e7-b0ce-c5052bd4aad5 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:52.723549 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-60df6b5f-a88d-46e7-b0ce-c5052bd4aad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:52.782550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:52.794965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 31/59] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:35:52.820187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6674fd-1b66-43cf-9b10-ce201a8ce5f8 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:52.886732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:52.933548 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6674fd-1b66-43cf-9b10-ce201a8ce5f8 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:52.973442 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ac81b-35cd-4d5e-8871-3dfc1d50bf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:52.974069 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ac81b-35cd-4d5e-8871-3dfc1d50bf28 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:52.974347 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ac81b-35cd-4d5e-8871-3dfc1d50bf28 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:52.974656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ac81b-35cd-4d5e-8871-3dfc1d50bf28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:52.975826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ac81b-35cd-4d5e-8871-3dfc1d50bf28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:52.977306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ac81b-35cd-4d5e-8871-3dfc1d50bf28 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:52.978937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 29/60] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:35:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:52.984702 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df6b5f-a88d-46e7-b0ce-c5052bd4aad5 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:53.085965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df6b5f-a88d-46e7-b0ce-c5052bd4aad5 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:53.120738 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6ffb9-83d1-4712-8635-3c21ba5f496a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:53.121366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6ffb9-83d1-4712-8635-3c21ba5f496a None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:53.121649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6ffb9-83d1-4712-8635-3c21ba5f496a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:53.121929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6ffb9-83d1-4712-8635-3c21ba5f496a None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:53.122645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6ffb9-83d1-4712-8635-3c21ba5f496a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:53.124119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6ffb9-83d1-4712-8635-3c21ba5f496a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:53.153574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 32/61] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:35:52 2022] POST /identity/v3/users => generated 275 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:35:56.594201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a963a2d-4d61-43a5-a699-eebfb0822da7 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:56.594201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a963a2d-4d61-43a5-a699-eebfb0822da7 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:56.594201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a963a2d-4d61-43a5-a699-eebfb0822da7 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:56.594201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 30/62] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:35:56.601165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545ed7a-69d3-4199-9dc3-88d179689419 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:56.602748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545ed7a-69d3-4199-9dc3-88d179689419 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:56.603404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545ed7a-69d3-4199-9dc3-88d179689419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:56.624384 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c545ed7a-69d3-4199-9dc3-88d179689419 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:56.625933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c545ed7a-69d3-4199-9dc3-88d179689419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:56.655651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f720531-354c-42a1-a476-4554550a4316 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:56.656573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f720531-354c-42a1-a476-4554550a4316 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:56.657302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f720531-354c-42a1-a476-4554550a4316 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:56.660501 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 31/63] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:56.670060 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39963cf9-e62f-49d5-9f25-844ca717a6aa None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:56.670975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39963cf9-e62f-49d5-9f25-844ca717a6aa None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:56.671712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39963cf9-e62f-49d5-9f25-844ca717a6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:56.687136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:56.720610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-39963cf9-e62f-49d5-9f25-844ca717a6aa None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:56.723032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-39963cf9-e62f-49d5-9f25-844ca717a6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:56.777216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 33/64] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:35:56.823121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545ed7a-69d3-4199-9dc3-88d179689419 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:56.838791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 32/65] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:35:56.875799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39963cf9-e62f-49d5-9f25-844ca717a6aa None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:57.027174 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:57.094485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545ed7a-69d3-4199-9dc3-88d179689419 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:57.131587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39963cf9-e62f-49d5-9f25-844ca717a6aa None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:57.193071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79417804-5434-4aab-918f-de37dd456058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:57.193733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79417804-5434-4aab-918f-de37dd456058 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:57.194107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79417804-5434-4aab-918f-de37dd456058 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:57.194452 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79417804-5434-4aab-918f-de37dd456058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:57.213017 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79417804-5434-4aab-918f-de37dd456058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:35:57.214380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74d665-436d-4dce-bc27-5b6686ab2374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:57.215776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74d665-436d-4dce-bc27-5b6686ab2374 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:57.216502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74d665-436d-4dce-bc27-5b6686ab2374 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:57.217048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74d665-436d-4dce-bc27-5b6686ab2374 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-79417804-5434-4aab-918f-de37dd456058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:35:57.218960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:35:57.219959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:35:57.221062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:35:57.221062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:35:57.221062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:35:57.221062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:35:57.221062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:35:57.221062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:35:57.221062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:35:57.221062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:35:57.221062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:35:57.221062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:35:57.221062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:35:57.221062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:35:57.221062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 34/66] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:35:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:35:57.222727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c6cb6-779b-4ca4-8ff5-8c68c6a67418 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:57.222878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c6cb6-779b-4ca4-8ff5-8c68c6a67418 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:57.223144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c6cb6-779b-4ca4-8ff5-8c68c6a67418 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:57.225129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 35/67] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:57.232500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c6cb6-779b-4ca4-8ff5-8c68c6a67418 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:57.237942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74d665-436d-4dce-bc27-5b6686ab2374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-bf74d665-436d-4dce-bc27-5b6686ab2374 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:35:57.246686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:35:57.247773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:35:57.248671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:35:57.248671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:35:57.248671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:35:57.248671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:35:57.248671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:35:57.248671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:35:57.248671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:35:57.248671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:35:57.248671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:35:57.248671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:35:57.248671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:35:57.248671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:35:57.251198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 33/68] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:35:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:35:57.252965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7ce90-460a-41fd-b758-94a8c8168d90 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:57.253539 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7ce90-460a-41fd-b758-94a8c8168d90 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:57.254053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7ce90-460a-41fd-b758-94a8c8168d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:57.280720 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-24d7ce90-460a-41fd-b758-94a8c8168d90 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:57.283149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-24d7ce90-460a-41fd-b758-94a8c8168d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:57.285777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:57.406809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 34/69] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:35:57.408327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7ce90-460a-41fd-b758-94a8c8168d90 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:57.421518 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c6cb6-779b-4ca4-8ff5-8c68c6a67418 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:57.481096 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b08152-1065-4749-9d81-7a5f8ee7bde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:57.481749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b08152-1065-4749-9d81-7a5f8ee7bde6 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:57.482031 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b08152-1065-4749-9d81-7a5f8ee7bde6 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:57.482307 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b08152-1065-4749-9d81-7a5f8ee7bde6 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:57.483117 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b08152-1065-4749-9d81-7a5f8ee7bde6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:35:57.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b08152-1065-4749-9d81-7a5f8ee7bde6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:57.485411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b08152-1065-4749-9d81-7a5f8ee7bde6 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:57.497615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 36/70] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:35:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:57.502296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d42d7a-1c9d-4235-ae4b-e5d7a4f04afa None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:57.502296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d42d7a-1c9d-4235-ae4b-e5d7a4f04afa None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:57.502296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d42d7a-1c9d-4235-ae4b-e5d7a4f04afa None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:57.502296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 37/71] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:57.502296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d42d7a-1c9d-4235-ae4b-e5d7a4f04afa None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:57.514392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:57.583093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7ce90-460a-41fd-b758-94a8c8168d90 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:57.638298 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2c8b7-5298-46a3-84a1-b143cd01ca71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:57.639224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2c8b7-5298-46a3-84a1-b143cd01ca71 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:57.639224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2c8b7-5298-46a3-84a1-b143cd01ca71 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:57.639488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2c8b7-5298-46a3-84a1-b143cd01ca71 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:57.640224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2c8b7-5298-46a3-84a1-b143cd01ca71 None admin] RBAC: Adding query filter params (name=member) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:35:57.640529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2c8b7-5298-46a3-84a1-b143cd01ca71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:57.641951 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2c8b7-5298-46a3-84a1-b143cd01ca71 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:57.652814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 35/72] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:35:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:57.652814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2c8b7-5298-46a3-84a1-b143cd01ca71 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:57.691760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:57.699703 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d42d7a-1c9d-4235-ae4b-e5d7a4f04afa None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:57.753054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54f20d-29c3-4fa0-97bd-ec676311ba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:57.753712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54f20d-29c3-4fa0-97bd-ec676311ba13 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:57.753996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54f20d-29c3-4fa0-97bd-ec676311ba13 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:57.754267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54f20d-29c3-4fa0-97bd-ec676311ba13 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:57.767428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54f20d-29c3-4fa0-97bd-ec676311ba13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-9a54f20d-29c3-4fa0-97bd-ec676311ba13 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:35:57.775297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:35:57.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 38/73] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:35:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:35:57.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca8649-2fd5-44a6-9c49-eaaf863d90bc None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:57.779795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca8649-2fd5-44a6-9c49-eaaf863d90bc None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:57.779795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca8649-2fd5-44a6-9c49-eaaf863d90bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:57.801506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-bfca8649-2fd5-44a6-9c49-eaaf863d90bc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:57.801506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-bfca8649-2fd5-44a6-9c49-eaaf863d90bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:57.801998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2c8b7-5298-46a3-84a1-b143cd01ca71 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:57.816211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:57.849802 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9a198-1947-4083-b83c-daa25db15dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:57.850460 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9a198-1947-4083-b83c-daa25db15dc9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:57.850772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9a198-1947-4083-b83c-daa25db15dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:57.851047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9a198-1947-4083-b83c-daa25db15dc9 None admin] PATH_INFO: `/v3/users/047b4b41672048d8889a2b3bb55b2d97` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:57.852863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9a198-1947-4083-b83c-daa25db15dc9 None admin] RBAC: Authorizing `identity:get_user(user_id=047b4b41672048d8889a2b3bb55b2d97)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:57.854315 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9a198-1947-4083-b83c-daa25db15dc9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:57.856007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 36/74] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:35:57 2022] GET /identity/v3/users/047b4b41672048d8889a2b3bb55b2d97 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:57.856792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9a198-1947-4083-b83c-daa25db15dc9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:57.886110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 39/75] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:35:57.887497 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8b084-437f-4ea9-a2b5-65433a7a093b None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:57.887785 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8b084-437f-4ea9-a2b5-65433a7a093b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:57.888057 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8b084-437f-4ea9-a2b5-65433a7a093b None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:57.889329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 40/76] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:35:57.890025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8b084-437f-4ea9-a2b5-65433a7a093b None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:57.990509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:57.994139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9a198-1947-4083-b83c-daa25db15dc9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:58.006275 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8b084-437f-4ea9-a2b5-65433a7a093b None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:58.037220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76f688-60c8-42f3-92a7-4345830b376d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:58.037870 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76f688-60c8-42f3-92a7-4345830b376d None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:58.038157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76f688-60c8-42f3-92a7-4345830b376d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:58.038434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76f688-60c8-42f3-92a7-4345830b376d None admin] PATH_INFO: `/v3/users/4b27046155b1457883bf610e3be6b70e` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:58.049076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a42c22-7d6e-4ef5-ad12-d11b017f19db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:58.049820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a42c22-7d6e-4ef5-ad12-d11b017f19db None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:58.050296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a42c22-7d6e-4ef5-ad12-d11b017f19db None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:58.050820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a42c22-7d6e-4ef5-ad12-d11b017f19db None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:58.051810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a42c22-7d6e-4ef5-ad12-d11b017f19db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:35:58.052239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a42c22-7d6e-4ef5-ad12-d11b017f19db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:58.054001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a42c22-7d6e-4ef5-ad12-d11b017f19db None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:58.057171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76f688-60c8-42f3-92a7-4345830b376d None admin] RBAC: Authorizing `identity:get_user(user_id=4b27046155b1457883bf610e3be6b70e)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:58.058688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76f688-60c8-42f3-92a7-4345830b376d None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:58.061126 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 37/77] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:35:57 2022] GET /identity/v3/users/4b27046155b1457883bf610e3be6b70e => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:58.062228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76f688-60c8-42f3-92a7-4345830b376d None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:58.065349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 41/78] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:35:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:58.067471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102a526-9aab-48bc-9fab-387520a480e0 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:58.067961 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102a526-9aab-48bc-9fab-387520a480e0 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:58.068463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102a526-9aab-48bc-9fab-387520a480e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:58.092060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-8102a526-9aab-48bc-9fab-387520a480e0 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:35:58.093282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8102a526-9aab-48bc-9fab-387520a480e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:35:58.113703 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:58.177919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 42/79] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:35:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:35:58.178803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102a526-9aab-48bc-9fab-387520a480e0 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:58.195554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76f688-60c8-42f3-92a7-4345830b376d None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:58.234256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c4224-732a-4062-9564-f65d5a649a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:58.234953 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c4224-732a-4062-9564-f65d5a649a66 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:58.235237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c4224-732a-4062-9564-f65d5a649a66 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:58.235520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c4224-732a-4062-9564-f65d5a649a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:58.280733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:58.307963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102a526-9aab-48bc-9fab-387520a480e0 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:58.316705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c4224-732a-4062-9564-f65d5a649a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:58.319056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c4224-732a-4062-9564-f65d5a649a66 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:58.347727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27885d21-ec4d-4c87-9609-a8c7ad7db8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:58.348374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27885d21-ec4d-4c87-9609-a8c7ad7db8b6 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:58.348664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27885d21-ec4d-4c87-9609-a8c7ad7db8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:58.348937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27885d21-ec4d-4c87-9609-a8c7ad7db8b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:58.363751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27885d21-ec4d-4c87-9609-a8c7ad7db8b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-27885d21-ec4d-4c87-9609-a8c7ad7db8b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:35:58.369047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:35:58.369955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 43/80] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:35:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:35:58.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27885d21-ec4d-4c87-9609-a8c7ad7db8b6 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:58.380362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:58.453564 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 38/81] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:35:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:35:58.454420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c4224-732a-4062-9564-f65d5a649a66 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:58.504977 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27885d21-ec4d-4c87-9609-a8c7ad7db8b6 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:58.550319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff8973-ea9f-4b7d-a236-3b9bbc128393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:58.550885 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff8973-ea9f-4b7d-a236-3b9bbc128393 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:58.551169 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff8973-ea9f-4b7d-a236-3b9bbc128393 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:58.551448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff8973-ea9f-4b7d-a236-3b9bbc128393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:58.564328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:58.571568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c4224-732a-4062-9564-f65d5a649a66 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:58.609141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba29edf-e5ff-4b10-a935-a8f24e8aafa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:58.609783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba29edf-e5ff-4b10-a935-a8f24e8aafa3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:58.610057 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba29edf-e5ff-4b10-a935-a8f24e8aafa3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:58.610329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba29edf-e5ff-4b10-a935-a8f24e8aafa3 None admin] PATH_INFO: `/v3/users/4b27046155b1457883bf610e3be6b70e` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:58.611551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba29edf-e5ff-4b10-a935-a8f24e8aafa3 None admin] RBAC: Authorizing `identity:get_user(user_id=4b27046155b1457883bf610e3be6b70e)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:58.613001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba29edf-e5ff-4b10-a935-a8f24e8aafa3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:58.614636 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 39/82] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:35:58 2022] GET /identity/v3/users/4b27046155b1457883bf610e3be6b70e => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:58.615421 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba29edf-e5ff-4b10-a935-a8f24e8aafa3 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:58.627680 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:58.632715 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff8973-ea9f-4b7d-a236-3b9bbc128393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:58.635212 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff8973-ea9f-4b7d-a236-3b9bbc128393 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:58.723933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba29edf-e5ff-4b10-a935-a8f24e8aafa3 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:58.752294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 44/83] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:35:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:35:58.753077 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff8973-ea9f-4b7d-a236-3b9bbc128393 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:58.763974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad9d78-1a1d-497d-95d4-f2c2e26a3ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:58.764617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad9d78-1a1d-497d-95d4-f2c2e26a3ecf None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:58.764903 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad9d78-1a1d-497d-95d4-f2c2e26a3ecf None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:58.765177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad9d78-1a1d-497d-95d4-f2c2e26a3ecf None admin] PATH_INFO: `/v3/domains/default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:58.766401 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad9d78-1a1d-497d-95d4-f2c2e26a3ecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:58.767907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad9d78-1a1d-497d-95d4-f2c2e26a3ecf None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:58.769539 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 40/84] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:35:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:58.770308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad9d78-1a1d-497d-95d4-f2c2e26a3ecf None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:58.800730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:58.864080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:58.874010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff8973-ea9f-4b7d-a236-3b9bbc128393 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:58.890949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad9d78-1a1d-497d-95d4-f2c2e26a3ecf None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:58.918782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0d6d8-a5a5-4e71-803c-c3140f514d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:58.919451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0d6d8-a5a5-4e71-803c-c3140f514d7f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:58.919731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0d6d8-a5a5-4e71-803c-c3140f514d7f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:58.920002 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0d6d8-a5a5-4e71-803c-c3140f514d7f None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:58.920722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0d6d8-a5a5-4e71-803c-c3140f514d7f None admin] RBAC: Adding query filter params (name=member) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:35:58.921031 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0d6d8-a5a5-4e71-803c-c3140f514d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:58.922445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0d6d8-a5a5-4e71-803c-c3140f514d7f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:58.927454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8a05d-e5b6-4a63-8342-1ed5a1b50abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:58.928107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8a05d-e5b6-4a63-8342-1ed5a1b50abd None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:58.928391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8a05d-e5b6-4a63-8342-1ed5a1b50abd None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:58.928669 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8a05d-e5b6-4a63-8342-1ed5a1b50abd None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:58.929886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8a05d-e5b6-4a63-8342-1ed5a1b50abd None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:58.932267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8a05d-e5b6-4a63-8342-1ed5a1b50abd None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:58.934174 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 41/85] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:35:58 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:58.934500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 45/86] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:35:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:58.935090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8a05d-e5b6-4a63-8342-1ed5a1b50abd None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:58.935741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0d6d8-a5a5-4e71-803c-c3140f514d7f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:59.060020 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8a05d-e5b6-4a63-8342-1ed5a1b50abd None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:59.068005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0d6d8-a5a5-4e71-803c-c3140f514d7f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:59.070490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:59.103465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f104898-9978-4414-a5f6-2a6bfe72ca89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:59.104086 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f104898-9978-4414-a5f6-2a6bfe72ca89 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:59.104367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f104898-9978-4414-a5f6-2a6bfe72ca89 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:59.104648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f104898-9978-4414-a5f6-2a6bfe72ca89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:59.111542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8945093-bf3f-4c5a-85df-74ba5ec7fddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:59.112481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8945093-bf3f-4c5a-85df-74ba5ec7fddb None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:59.112903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8945093-bf3f-4c5a-85df-74ba5ec7fddb None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:59.113326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8945093-bf3f-4c5a-85df-74ba5ec7fddb None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:59.115235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8945093-bf3f-4c5a-85df-74ba5ec7fddb None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:59.118165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8945093-bf3f-4c5a-85df-74ba5ec7fddb None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:59.120782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 46/87] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:35:58 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:59.121325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:59.121553 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8945093-bf3f-4c5a-85df-74ba5ec7fddb None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:59.203142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f104898-9978-4414-a5f6-2a6bfe72ca89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:59.204609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f104898-9978-4414-a5f6-2a6bfe72ca89 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:59.250516 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8945093-bf3f-4c5a-85df-74ba5ec7fddb None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:59.292140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcb33c-93be-456e-a69b-3e90d1182639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:59.292776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcb33c-93be-456e-a69b-3e90d1182639 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:59.293059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcb33c-93be-456e-a69b-3e90d1182639 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:59.293341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcb33c-93be-456e-a69b-3e90d1182639 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:59.294036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abcb33c-93be-456e-a69b-3e90d1182639 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:59.295546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abcb33c-93be-456e-a69b-3e90d1182639 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:59.318812 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 47/88] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:35:59 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:35:59.319569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcb33c-93be-456e-a69b-3e90d1182639 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:59.340490 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 42/89] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:35:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:35:59.340490 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f104898-9978-4414-a5f6-2a6bfe72ca89 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:59.377849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:59.409488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:59.500765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcb33c-93be-456e-a69b-3e90d1182639 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:59.503157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f104898-9978-4414-a5f6-2a6bfe72ca89 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:59.551631 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14dc94-97a6-4cf6-a858-1efb46799fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:59.551631 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14dc94-97a6-4cf6-a858-1efb46799fe8 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:59.551631 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14dc94-97a6-4cf6-a858-1efb46799fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:59.551631 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14dc94-97a6-4cf6-a858-1efb46799fe8 None admin] PATH_INFO: `/v3/users/4b27046155b1457883bf610e3be6b70e` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:59.552884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14dc94-97a6-4cf6-a858-1efb46799fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=4b27046155b1457883bf610e3be6b70e)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:59.554331 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14dc94-97a6-4cf6-a858-1efb46799fe8 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:59.556897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 43/90] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:35:59 2022] GET /identity/v3/users/4b27046155b1457883bf610e3be6b70e => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:59.558260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14dc94-97a6-4cf6-a858-1efb46799fe8 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:59.559541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c201470-286f-4379-acda-a2248d7762ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:59.559733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c201470-286f-4379-acda-a2248d7762ea None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:59.560027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c201470-286f-4379-acda-a2248d7762ea None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:59.560309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c201470-286f-4379-acda-a2248d7762ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:59.561132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c201470-286f-4379-acda-a2248d7762ea None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=11d6ae35861149a9a30b9eadb5380b85, user.id=047b4b41672048d8889a2b3bb55b2d97) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:35:59.561451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c201470-286f-4379-acda-a2248d7762ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:59.563263 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c201470-286f-4379-acda-a2248d7762ea None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:59.569842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 48/91] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:35:59 2022] GET /identity/v3/role_assignments?user.id=047b4b41672048d8889a2b3bb55b2d97&scope.project.id=11d6ae35861149a9a30b9eadb5380b85&role.id=93eb204498fc49d297bcd1841e00818d => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:59.572041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c201470-286f-4379-acda-a2248d7762ea None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:59.719835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14dc94-97a6-4cf6-a858-1efb46799fe8 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:59.727776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:59.745905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c201470-286f-4379-acda-a2248d7762ea None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:35:59.751640 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:35:59.793044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324dc4b-43d2-42bb-98de-a05bcf14fbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:59.794126 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324dc4b-43d2-42bb-98de-a05bcf14fbd3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:59.794650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324dc4b-43d2-42bb-98de-a05bcf14fbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:59.795227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324dc4b-43d2-42bb-98de-a05bcf14fbd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:59.796395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324dc4b-43d2-42bb-98de-a05bcf14fbd3 None admin] RBAC: Adding query filter params (role.id=fe6c7625134a449c8395ab84ffd82371, scope.project.id=11d6ae35861149a9a30b9eadb5380b85, user.id=4b27046155b1457883bf610e3be6b70e) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:35:59.797592 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324dc4b-43d2-42bb-98de-a05bcf14fbd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:59.799719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324dc4b-43d2-42bb-98de-a05bcf14fbd3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:59.801115 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551af9e-ce41-4f55-a93e-ab9bbc43333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:35:59.801773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551af9e-ce41-4f55-a93e-ab9bbc43333b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:35:59.802054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551af9e-ce41-4f55-a93e-ab9bbc43333b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:35:59.802713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551af9e-ce41-4f55-a93e-ab9bbc43333b None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:35:59.803985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551af9e-ce41-4f55-a93e-ab9bbc43333b None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:35:59.805688 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551af9e-ce41-4f55-a93e-ab9bbc43333b None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:35:59.809500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 49/92] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:35:59 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:59.810179 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 44/93] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:35:59 2022] GET /identity/v3/role_assignments?user.id=4b27046155b1457883bf610e3be6b70e&scope.project.id=11d6ae35861149a9a30b9eadb5380b85&role.id=fe6c7625134a449c8395ab84ffd82371 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:35:59.810833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551af9e-ce41-4f55-a93e-ab9bbc43333b None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:35:59.816659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324dc4b-43d2-42bb-98de-a05bcf14fbd3 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:00.078221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551af9e-ce41-4f55-a93e-ab9bbc43333b None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:00.086044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324dc4b-43d2-42bb-98de-a05bcf14fbd3 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:00.163960 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2d574-f25d-4c13-a20c-7d88f2c12ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:00.165246 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2d574-f25d-4c13-a20c-7d88f2c12ec0 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:00.165865 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2d574-f25d-4c13-a20c-7d88f2c12ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:00.166360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09380c6-e2e6-4eb2-931e-61544cd638f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:00.166519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2d574-f25d-4c13-a20c-7d88f2c12ec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:00.167027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09380c6-e2e6-4eb2-931e-61544cd638f3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:00.167214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2d574-f25d-4c13-a20c-7d88f2c12ec0 None admin] RBAC: Adding query filter params (role.id=686764f0686e446f884315081e158bf3, scope.project.id=11d6ae35861149a9a30b9eadb5380b85, user.id=4b27046155b1457883bf610e3be6b70e) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:00.167309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09380c6-e2e6-4eb2-931e-61544cd638f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:00.167527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2d574-f25d-4c13-a20c-7d88f2c12ec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:00.167623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09380c6-e2e6-4eb2-931e-61544cd638f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:00.169209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2d574-f25d-4c13-a20c-7d88f2c12ec0 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:00.186173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 45/94] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:35:59 2022] GET /identity/v3/role_assignments?user.id=4b27046155b1457883bf610e3be6b70e&scope.project.id=11d6ae35861149a9a30b9eadb5380b85&role.id=686764f0686e446f884315081e158bf3 => generated 259 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:00.222146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:00.290708 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09380c6-e2e6-4eb2-931e-61544cd638f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:00.292267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09380c6-e2e6-4eb2-931e-61544cd638f3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:00.438242 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 50/95] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:35:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:00.443952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2d574-f25d-4c13-a20c-7d88f2c12ec0 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:00.501611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:00.584737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2d574-f25d-4c13-a20c-7d88f2c12ec0 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:00.634138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e3735-5bf9-4777-bc4d-d9ad3805e142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:00.635367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e3735-5bf9-4777-bc4d-d9ad3805e142 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:00.635838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e3735-5bf9-4777-bc4d-d9ad3805e142 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:00.636322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e3735-5bf9-4777-bc4d-d9ad3805e142 None admin] PATH_INFO: `/v3/projects/f13fcc67893d41a2af55704f6f81b1d4` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:00.638655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e3735-5bf9-4777-bc4d-d9ad3805e142 None admin] RBAC: Authorizing `identity:get_project(project_id=f13fcc67893d41a2af55704f6f81b1d4)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:00.641004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e3735-5bf9-4777-bc4d-d9ad3805e142 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:00.642864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 46/96] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:00 2022] GET /identity/v3/projects/f13fcc67893d41a2af55704f6f81b1d4 => generated 312 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:00.650137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09380c6-e2e6-4eb2-931e-61544cd638f3 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:00.734366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:00.790925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09380c6-e2e6-4eb2-931e-61544cd638f3 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:00.842454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f77e54-f5ce-4fc1-b5b8-c6b5169d58ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:00.843439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f77e54-f5ce-4fc1-b5b8-c6b5169d58ec None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:00.843439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f77e54-f5ce-4fc1-b5b8-c6b5169d58ec None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:00.843669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f77e54-f5ce-4fc1-b5b8-c6b5169d58ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:00.844429 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f77e54-f5ce-4fc1-b5b8-c6b5169d58ec None admin] RBAC: Adding query filter params (role.id=fe6c7625134a449c8395ab84ffd82371, scope.project.id=f13fcc67893d41a2af55704f6f81b1d4, user.id=4b27046155b1457883bf610e3be6b70e) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:00.844736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f77e54-f5ce-4fc1-b5b8-c6b5169d58ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:00.846343 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f77e54-f5ce-4fc1-b5b8-c6b5169d58ec None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:00.853346 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 51/97] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:00 2022] GET /identity/v3/role_assignments?user.id=4b27046155b1457883bf610e3be6b70e&scope.project.id=f13fcc67893d41a2af55704f6f81b1d4&role.id=fe6c7625134a449c8395ab84ffd82371 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:02.817906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26465fef-a177-4a1e-8f31-dd5621fde847 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:02.819037 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26465fef-a177-4a1e-8f31-dd5621fde847 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:02.819554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26465fef-a177-4a1e-8f31-dd5621fde847 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:02.822087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 47/98] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:02.828531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7baff13-17f6-4739-8d18-ea49e9f7b529 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:02.830005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7baff13-17f6-4739-8d18-ea49e9f7b529 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:02.830791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7baff13-17f6-4739-8d18-ea49e9f7b529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:02.854990 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-b7baff13-17f6-4739-8d18-ea49e9f7b529 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:02.857439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b7baff13-17f6-4739-8d18-ea49e9f7b529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:02.943286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 52/99] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:02.956972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a836f-b002-4481-a70b-9bf64e5e2e06 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:02.957585 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a836f-b002-4481-a70b-9bf64e5e2e06 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:02.958062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a836f-b002-4481-a70b-9bf64e5e2e06 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:02.959974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 48/100] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:02.965877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd1487-8844-4ccc-be01-00d7cdc4061b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:02.966463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd1487-8844-4ccc-be01-00d7cdc4061b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:02.966953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd1487-8844-4ccc-be01-00d7cdc4061b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:02.976195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a836f-b002-4481-a70b-9bf64e5e2e06 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:02.995543 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3dcd1487-8844-4ccc-be01-00d7cdc4061b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:02.996757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3dcd1487-8844-4ccc-be01-00d7cdc4061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:03.120354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:03.155228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a836f-b002-4481-a70b-9bf64e5e2e06 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:03.167539 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:03.174726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 53/101] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:03.176497 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5e527-fbba-464f-9f01-1fb837643d3b None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:03.176970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5e527-fbba-464f-9f01-1fb837643d3b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:03.177489 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5e527-fbba-464f-9f01-1fb837643d3b None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:03.179178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 54/102] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:03.185439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d550cc-2756-4f2a-8455-a5e4c7cfbd5f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:03.186110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d550cc-2756-4f2a-8455-a5e4c7cfbd5f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:03.186583 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d550cc-2756-4f2a-8455-a5e4c7cfbd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:03.220755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d45cfe-901d-4313-b951-960f9a4c771a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:03.222143 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d45cfe-901d-4313-b951-960f9a4c771a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:03.222828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d45cfe-901d-4313-b951-960f9a4c771a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:03.223488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d45cfe-901d-4313-b951-960f9a4c771a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:03.224145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e7d550cc-2756-4f2a-8455-a5e4c7cfbd5f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:03.225291 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d45cfe-901d-4313-b951-960f9a4c771a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:03.225585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e7d550cc-2756-4f2a-8455-a5e4c7cfbd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:03.227675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d45cfe-901d-4313-b951-960f9a4c771a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:03.229542 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 49/103] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:03.236289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d45cfe-901d-4313-b951-960f9a4c771a None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:03.370998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 55/104] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:03.373444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d550cc-2756-4f2a-8455-a5e4c7cfbd5f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:03.393145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d45cfe-901d-4313-b951-960f9a4c771a None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:03.432609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadeb3b-d244-4311-b28b-1e33581201f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:03.433792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadeb3b-d244-4311-b28b-1e33581201f8 None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:03.434226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadeb3b-d244-4311-b28b-1e33581201f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:03.434690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadeb3b-d244-4311-b28b-1e33581201f8 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:03.435715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadeb3b-d244-4311-b28b-1e33581201f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:03.438100 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadeb3b-d244-4311-b28b-1e33581201f8 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:03.496153 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 50/105] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:36:03 2022] POST /identity/v3/projects => generated 302 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:36:03.498940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58cf0b-3fd0-4325-b475-0b20b89e42e6 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:03.499751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58cf0b-3fd0-4325-b475-0b20b89e42e6 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:03.500501 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58cf0b-3fd0-4325-b475-0b20b89e42e6 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:03.503500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 51/106] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:03.512106 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58cf0b-3fd0-4325-b475-0b20b89e42e6 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:03.593275 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d550cc-2756-4f2a-8455-a5e4c7cfbd5f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:03.619948 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:03.647621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:03.722743 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263403d-3b12-424c-8ef6-df7579767d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:03.723431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263403d-3b12-424c-8ef6-df7579767d31 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:03.723716 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263403d-3b12-424c-8ef6-df7579767d31 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:03.723990 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263403d-3b12-424c-8ef6-df7579767d31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:03.738247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263403d-3b12-424c-8ef6-df7579767d31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-5263403d-3b12-424c-8ef6-df7579767d31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:03.745203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:03.746116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:03.746944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:03.746944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:03.746944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:03.746944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:03.746944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:03.746944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:03.746944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:03.746944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:03.746944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:03.746944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:03.746944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:03.746944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:03.746944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 56/107] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:03.748354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86ea3e-c00f-4660-a9f3-ea7acd460145 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:03.749727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86ea3e-c00f-4660-a9f3-ea7acd460145 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:03.749727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86ea3e-c00f-4660-a9f3-ea7acd460145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:03.785676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-4f86ea3e-c00f-4660-a9f3-ea7acd460145 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:03.787659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4f86ea3e-c00f-4660-a9f3-ea7acd460145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:03.805920 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58cf0b-3fd0-4325-b475-0b20b89e42e6 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:03.865985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302ea47-aa2b-4e7d-a6f0-e401f4451b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:03.866674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302ea47-aa2b-4e7d-a6f0-e401f4451b1a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:03.867060 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302ea47-aa2b-4e7d-a6f0-e401f4451b1a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:03.867332 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302ea47-aa2b-4e7d-a6f0-e401f4451b1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:03.879906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302ea47-aa2b-4e7d-a6f0-e401f4451b1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-e302ea47-aa2b-4e7d-a6f0-e401f4451b1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:03.885555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:03.887167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:03.888341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:03.888341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:03.888341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:03.888341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:03.888341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:03.888341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:03.888341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:03.888341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:03.888341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:03.888341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:03.888341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:03.888341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:03.888341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 52/108] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:36:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:03.890097 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef72e0c-6f1d-4caf-9423-0eb89385a7f9 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:03.890097 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef72e0c-6f1d-4caf-9423-0eb89385a7f9 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:03.890097 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef72e0c-6f1d-4caf-9423-0eb89385a7f9 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:03.890569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 53/109] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:03.895734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef72e0c-6f1d-4caf-9423-0eb89385a7f9 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:03.906011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 57/110] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:03.908278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3d0cd-e256-48d5-a29d-f8385eb51da4 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:03.908801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3d0cd-e256-48d5-a29d-f8385eb51da4 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:03.909277 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3d0cd-e256-48d5-a29d-f8385eb51da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:03.929546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-f3f3d0cd-e256-48d5-a29d-f8385eb51da4 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:03.932067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f3f3d0cd-e256-48d5-a29d-f8385eb51da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:03.999021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:04.015009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef72e0c-6f1d-4caf-9423-0eb89385a7f9 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:04.019580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 58/111] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:04.020848 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3d0cd-e256-48d5-a29d-f8385eb51da4 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:04.061645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:04.086150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77801de6-5629-4727-a9c6-bfa95e1157f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:04.087377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77801de6-5629-4727-a9c6-bfa95e1157f5 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:04.087657 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77801de6-5629-4727-a9c6-bfa95e1157f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:04.087937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77801de6-5629-4727-a9c6-bfa95e1157f5 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:04.088727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77801de6-5629-4727-a9c6-bfa95e1157f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:04.089032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77801de6-5629-4727-a9c6-bfa95e1157f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:04.090428 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77801de6-5629-4727-a9c6-bfa95e1157f5 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:04.099827 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 54/112] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:04.100676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77801de6-5629-4727-a9c6-bfa95e1157f5 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:04.165957 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3d0cd-e256-48d5-a29d-f8385eb51da4 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:04.225009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ec728-fc89-43f2-8727-63c563daf117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:04.225987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ec728-fc89-43f2-8727-63c563daf117 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:04.226575 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ec728-fc89-43f2-8727-63c563daf117 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:04.227434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ec728-fc89-43f2-8727-63c563daf117 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:04.228936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ec728-fc89-43f2-8727-63c563daf117 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:04.229448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ec728-fc89-43f2-8727-63c563daf117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:04.231843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ec728-fc89-43f2-8727-63c563daf117 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:04.246660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 59/113] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:04.248188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ec728-fc89-43f2-8727-63c563daf117 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:04.273013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77801de6-5629-4727-a9c6-bfa95e1157f5 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:04.312183 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1c7d1-7e64-4b72-af50-e954d8b6cafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:04.313136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1c7d1-7e64-4b72-af50-e954d8b6cafd None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:04.313428 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1c7d1-7e64-4b72-af50-e954d8b6cafd None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:04.313764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1c7d1-7e64-4b72-af50-e954d8b6cafd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:04.326500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e1c7d1-7e64-4b72-af50-e954d8b6cafd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-99e1c7d1-7e64-4b72-af50-e954d8b6cafd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:04.332408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:04.333309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:04.334099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:04.334099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:04.334099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:04.334099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:04.334099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:04.334099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:04.334099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:04.334099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:04.334099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:04.334099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:04.334099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:04.334099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:04.334099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 55/114] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:04.334797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1c7d1-7e64-4b72-af50-e954d8b6cafd None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:04.349641 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:04.368348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:04.375989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ec728-fc89-43f2-8727-63c563daf117 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:04.416734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974259fe-931e-4397-a932-62d05e308c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:04.417301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974259fe-931e-4397-a932-62d05e308c49 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:04.417586 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974259fe-931e-4397-a932-62d05e308c49 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:04.417861 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974259fe-931e-4397-a932-62d05e308c49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:04.429704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974259fe-931e-4397-a932-62d05e308c49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-974259fe-931e-4397-a932-62d05e308c49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:04.435650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:04.436560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:04.437412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:04.437412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:04.437412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:04.437412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:04.437412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:04.437412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:04.437412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:04.437412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:04.437412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:04.437412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:04.437412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:04.437412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:04.437412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 60/115] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:04.438679 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b16c30-50a5-4746-a368-0d2b2a912571 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:04.438931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b16c30-50a5-4746-a368-0d2b2a912571 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:04.439252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b16c30-50a5-4746-a368-0d2b2a912571 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:04.440683 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 61/116] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:04.441291 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b16c30-50a5-4746-a368-0d2b2a912571 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:04.468033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1c7d1-7e64-4b72-af50-e954d8b6cafd None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:04.513423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ce79e-44f4-4ca2-a061-d92b4a3f884f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:04.514152 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ce79e-44f4-4ca2-a061-d92b4a3f884f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:04.514358 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ce79e-44f4-4ca2-a061-d92b4a3f884f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:04.514683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035ce79e-44f4-4ca2-a061-d92b4a3f884f None admin] PATH_INFO: `/v3/users/4b27046155b1457883bf610e3be6b70e` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:04.515918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ce79e-44f4-4ca2-a061-d92b4a3f884f None admin] RBAC: Authorizing `identity:get_user(user_id=4b27046155b1457883bf610e3be6b70e)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:04.518524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035ce79e-44f4-4ca2-a061-d92b4a3f884f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:04.520837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 56/117] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:04 2022] GET /identity/v3/users/4b27046155b1457883bf610e3be6b70e => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:04.526289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ce79e-44f4-4ca2-a061-d92b4a3f884f None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:04.601022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b16c30-50a5-4746-a368-0d2b2a912571 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:04.640450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fad6cc-449c-4708-9908-44b807464eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:04.641104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fad6cc-449c-4708-9908-44b807464eac None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:04.641391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fad6cc-449c-4708-9908-44b807464eac None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:04.641670 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fad6cc-449c-4708-9908-44b807464eac None admin] PATH_INFO: `/v3/users/047b4b41672048d8889a2b3bb55b2d97` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:04.642905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fad6cc-449c-4708-9908-44b807464eac None admin] RBAC: Authorizing `identity:get_user(user_id=047b4b41672048d8889a2b3bb55b2d97)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:04.644274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:04.644386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fad6cc-449c-4708-9908-44b807464eac None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:04.646032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 62/118] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:04 2022] GET /identity/v3/users/047b4b41672048d8889a2b3bb55b2d97 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:04.646844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fad6cc-449c-4708-9908-44b807464eac None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:04.674752 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035ce79e-44f4-4ca2-a061-d92b4a3f884f None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:04.675560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:04.723639 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0c588-c682-4d60-94df-bd7e740a9f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:04.724287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0c588-c682-4d60-94df-bd7e740a9f57 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:04.724587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0c588-c682-4d60-94df-bd7e740a9f57 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:04.724866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0c588-c682-4d60-94df-bd7e740a9f57 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:04.725624 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0c588-c682-4d60-94df-bd7e740a9f57 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:04.725937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0c588-c682-4d60-94df-bd7e740a9f57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:04.727467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0c588-c682-4d60-94df-bd7e740a9f57 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:04.745012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 57/119] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:04.745012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55514d92-c344-40ba-b04b-fbfe10d2db32 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:04.745012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55514d92-c344-40ba-b04b-fbfe10d2db32 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:04.745012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55514d92-c344-40ba-b04b-fbfe10d2db32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:04.759864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-55514d92-c344-40ba-b04b-fbfe10d2db32 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:04.761119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-55514d92-c344-40ba-b04b-fbfe10d2db32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:04.777947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fad6cc-449c-4708-9908-44b807464eac None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:04.819550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6f680-1ca7-4851-8de3-0715ad03caf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:04.820173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6f680-1ca7-4851-8de3-0715ad03caf5 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:04.820460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6f680-1ca7-4851-8de3-0715ad03caf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:04.821261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6f680-1ca7-4851-8de3-0715ad03caf5 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:04.822003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6f680-1ca7-4851-8de3-0715ad03caf5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:04.822400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6f680-1ca7-4851-8de3-0715ad03caf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:04.823839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6f680-1ca7-4851-8de3-0715ad03caf5 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:04.838564 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 63/120] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:04.838564 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6f680-1ca7-4851-8de3-0715ad03caf5 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:04.858821 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 58/121] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:04.859841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55514d92-c344-40ba-b04b-fbfe10d2db32 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:04.949257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:04.950881 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:04.968972 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6f680-1ca7-4851-8de3-0715ad03caf5 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:04.992243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55514d92-c344-40ba-b04b-fbfe10d2db32 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:05.009696 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabd267-b09a-425e-893d-f2cfbfa1b912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:05.010374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabd267-b09a-425e-893d-f2cfbfa1b912 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:05.010692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabd267-b09a-425e-893d-f2cfbfa1b912 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:05.010976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabd267-b09a-425e-893d-f2cfbfa1b912 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:05.012192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabd267-b09a-425e-893d-f2cfbfa1b912 None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:05.013654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabd267-b09a-425e-893d-f2cfbfa1b912 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:05.015677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 64/122] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:04 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:05.016206 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabd267-b09a-425e-893d-f2cfbfa1b912 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:05.051678 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d70e19-9784-487a-b98d-b4c4d171a4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:05.052385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d70e19-9784-487a-b98d-b4c4d171a4d8 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:05.052666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d70e19-9784-487a-b98d-b4c4d171a4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:05.052965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d70e19-9784-487a-b98d-b4c4d171a4d8 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:05.054158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d70e19-9784-487a-b98d-b4c4d171a4d8 None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:05.056192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d70e19-9784-487a-b98d-b4c4d171a4d8 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:05.058342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 59/123] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:04 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:05.058810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d70e19-9784-487a-b98d-b4c4d171a4d8 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:05.151435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabd267-b09a-425e-893d-f2cfbfa1b912 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:05.186034 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d70e19-9784-487a-b98d-b4c4d171a4d8 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:05.191390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d9574-a5e6-4fca-b56a-db6b6d9baf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:05.192039 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d9574-a5e6-4fca-b56a-db6b6d9baf04 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:05.192320 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d9574-a5e6-4fca-b56a-db6b6d9baf04 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:05.192646 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d9574-a5e6-4fca-b56a-db6b6d9baf04 None admin] PATH_INFO: `/v3/users/eebd684197654583a282fc9e4573075b` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:05.207543 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d9574-a5e6-4fca-b56a-db6b6d9baf04 None admin] RBAC: Authorizing `identity:get_user(user_id=eebd684197654583a282fc9e4573075b)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:05.208928 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d9574-a5e6-4fca-b56a-db6b6d9baf04 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:05.210700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 65/124] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:05 2022] GET /identity/v3/users/eebd684197654583a282fc9e4573075b => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:05.216890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d9574-a5e6-4fca-b56a-db6b6d9baf04 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:05.225237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2c9b3-aefd-4678-ad7f-27a02b8d9512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:05.226012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2c9b3-aefd-4678-ad7f-27a02b8d9512 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:05.226159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2c9b3-aefd-4678-ad7f-27a02b8d9512 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:05.226431 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2c9b3-aefd-4678-ad7f-27a02b8d9512 None admin] PATH_INFO: `/v3/users/4b27046155b1457883bf610e3be6b70e` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:05.228336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2c9b3-aefd-4678-ad7f-27a02b8d9512 None admin] RBAC: Authorizing `identity:get_user(user_id=4b27046155b1457883bf610e3be6b70e)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:05.229878 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2c9b3-aefd-4678-ad7f-27a02b8d9512 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:05.231127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:05.232255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 60/125] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:05 2022] GET /identity/v3/users/4b27046155b1457883bf610e3be6b70e => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:05.237249 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2c9b3-aefd-4678-ad7f-27a02b8d9512 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:05.246617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:05.333131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d9574-a5e6-4fca-b56a-db6b6d9baf04 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:05.359006 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2c9b3-aefd-4678-ad7f-27a02b8d9512 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:05.376920 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f57071-30d1-4264-a4f9-b38a66c0db4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:05.377820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57071-30d1-4264-a4f9-b38a66c0db4b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:05.378159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57071-30d1-4264-a4f9-b38a66c0db4b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:05.378543 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f57071-30d1-4264-a4f9-b38a66c0db4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:05.392382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f57071-30d1-4264-a4f9-b38a66c0db4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-66f57071-30d1-4264-a4f9-b38a66c0db4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:05.397477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:05.398377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 66/126] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:36:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:05.400013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f57071-30d1-4264-a4f9-b38a66c0db4b None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:05.422518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ac55-10d7-40f9-a186-63b0f7cdac70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:05.423327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ac55-10d7-40f9-a186-63b0f7cdac70 None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:05.423676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ac55-10d7-40f9-a186-63b0f7cdac70 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:05.424020 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176ac55-10d7-40f9-a186-63b0f7cdac70 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85/users/4b27046155b1457883bf610e3be6b70e/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:05.426222 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176ac55-10d7-40f9-a186-63b0f7cdac70 None admin] RBAC: Authorizing `identity:create_grant(project_id=11d6ae35861149a9a30b9eadb5380b85, user_id=4b27046155b1457883bf610e3be6b70e, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:05.427713 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176ac55-10d7-40f9-a186-63b0f7cdac70 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:05.458744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 61/127] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:36:05 2022] PUT /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85/users/4b27046155b1457883bf610e3be6b70e/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:36:05.458744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ac55-10d7-40f9-a186-63b0f7cdac70 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:05.501302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:05.545903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f57071-30d1-4264-a4f9-b38a66c0db4b None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:05.570352 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:05.589133 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc3028-bebe-4214-8e50-87184c262121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:05.590307 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc3028-bebe-4214-8e50-87184c262121 None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:05.590805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc3028-bebe-4214-8e50-87184c262121 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:05.591289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc3028-bebe-4214-8e50-87184c262121 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85/users/047b4b41672048d8889a2b3bb55b2d97/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:05.595031 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc3028-bebe-4214-8e50-87184c262121 None admin] RBAC: Authorizing `identity:create_grant(project_id=11d6ae35861149a9a30b9eadb5380b85, user_id=047b4b41672048d8889a2b3bb55b2d97, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:05.596854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc3028-bebe-4214-8e50-87184c262121 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:05.623041 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176ac55-10d7-40f9-a186-63b0f7cdac70 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:05.624311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 67/128] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:36:05 2022] PUT /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85/users/047b4b41672048d8889a2b3bb55b2d97/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:36:05.626051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc3028-bebe-4214-8e50-87184c262121 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:05.682667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a3b76-9a91-4993-b795-a6c1e1c64c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:05.683711 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a3b76-9a91-4993-b795-a6c1e1c64c56 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:05.684263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a3b76-9a91-4993-b795-a6c1e1c64c56 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:05.684784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a3b76-9a91-4993-b795-a6c1e1c64c56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:05.774998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc3028-bebe-4214-8e50-87184c262121 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:05.786486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a3b76-9a91-4993-b795-a6c1e1c64c56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:05.788509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a3b76-9a91-4993-b795-a6c1e1c64c56 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:05.827674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:05.835155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7f3a5-2699-4bac-81ea-efdfdedb5c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:05.835817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7f3a5-2699-4bac-81ea-efdfdedb5c5a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:05.836101 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7f3a5-2699-4bac-81ea-efdfdedb5c5a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:05.836896 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7f3a5-2699-4bac-81ea-efdfdedb5c5a None admin] PATH_INFO: `/v3/projects/f13fcc67893d41a2af55704f6f81b1d4` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:05.838502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7f3a5-2699-4bac-81ea-efdfdedb5c5a None admin] RBAC: Authorizing `identity:get_project(project_id=f13fcc67893d41a2af55704f6f81b1d4)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:05.840888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7f3a5-2699-4bac-81ea-efdfdedb5c5a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:05.843970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 68/129] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:05 2022] GET /identity/v3/projects/f13fcc67893d41a2af55704f6f81b1d4 => generated 312 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:05.845226 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7f3a5-2699-4bac-81ea-efdfdedb5c5a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:05.887242 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:05.944242 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 62/130] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:05.946914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa28d14-b50f-480c-87f5-46240ac211ba None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:05.947428 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa28d14-b50f-480c-87f5-46240ac211ba None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:05.947860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa28d14-b50f-480c-87f5-46240ac211ba None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:05.950117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 63/131] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:05.951462 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa28d14-b50f-480c-87f5-46240ac211ba None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:05.988664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7f3a5-2699-4bac-81ea-efdfdedb5c5a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:06.035120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a95abd-b7b2-41c1-a842-ea051a05dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:06.035671 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a95abd-b7b2-41c1-a842-ea051a05dfc6 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:06.035955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a95abd-b7b2-41c1-a842-ea051a05dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:06.036232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a95abd-b7b2-41c1-a842-ea051a05dfc6 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:06.036971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a95abd-b7b2-41c1-a842-ea051a05dfc6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:06.037262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a95abd-b7b2-41c1-a842-ea051a05dfc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:06.038682 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a95abd-b7b2-41c1-a842-ea051a05dfc6 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:06.049191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 69/132] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:36:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:06.050544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a95abd-b7b2-41c1-a842-ea051a05dfc6 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:06.098961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa28d14-b50f-480c-87f5-46240ac211ba None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:06.127438 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:06.147130 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3417594-7b9f-4ecf-abc1-19bcdf03c27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:06.148087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3417594-7b9f-4ecf-abc1-19bcdf03c27d None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:06.148527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3417594-7b9f-4ecf-abc1-19bcdf03c27d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:06.149730 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3417594-7b9f-4ecf-abc1-19bcdf03c27d None admin] PATH_INFO: `/v3/projects/f13fcc67893d41a2af55704f6f81b1d4/users/4b27046155b1457883bf610e3be6b70e/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:06.152032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3417594-7b9f-4ecf-abc1-19bcdf03c27d None admin] RBAC: Authorizing `identity:create_grant(project_id=f13fcc67893d41a2af55704f6f81b1d4, user_id=4b27046155b1457883bf610e3be6b70e, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:06.153658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3417594-7b9f-4ecf-abc1-19bcdf03c27d None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:06.178926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 64/133] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:36:05 2022] PUT /identity/v3/projects/f13fcc67893d41a2af55704f6f81b1d4/users/4b27046155b1457883bf610e3be6b70e/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:36:06.178926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60175393-8555-4c91-9fc1-77dc0ebefc03 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:06.178926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60175393-8555-4c91-9fc1-77dc0ebefc03 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:06.178926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60175393-8555-4c91-9fc1-77dc0ebefc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:06.200157 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a95abd-b7b2-41c1-a842-ea051a05dfc6 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:06.233336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-60175393-8555-4c91-9fc1-77dc0ebefc03 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:06.235527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-60175393-8555-4c91-9fc1-77dc0ebefc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:06.269739 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:06.286707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656c15d-fe9b-46b1-a6f4-075cd06090a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:06.288803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656c15d-fe9b-46b1-a6f4-075cd06090a3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:06.289209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656c15d-fe9b-46b1-a6f4-075cd06090a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:06.290140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656c15d-fe9b-46b1-a6f4-075cd06090a3 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:06.295707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656c15d-fe9b-46b1-a6f4-075cd06090a3 None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:06.298028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656c15d-fe9b-46b1-a6f4-075cd06090a3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:06.304675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 70/134] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:06 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:06.310809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656c15d-fe9b-46b1-a6f4-075cd06090a3 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:06.392743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 65/135] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:06.394150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60175393-8555-4c91-9fc1-77dc0ebefc03 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:06.491072 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656c15d-fe9b-46b1-a6f4-075cd06090a3 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:06.534189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:06.545433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b71b0d-44a6-4bda-9b1a-d32cb9183f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:06.546009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b71b0d-44a6-4bda-9b1a-d32cb9183f66 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:06.546293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b71b0d-44a6-4bda-9b1a-d32cb9183f66 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:06.546577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b71b0d-44a6-4bda-9b1a-d32cb9183f66 None admin] PATH_INFO: `/v3/users/4b27046155b1457883bf610e3be6b70e` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:06.547795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b71b0d-44a6-4bda-9b1a-d32cb9183f66 None admin] RBAC: Authorizing `identity:get_user(user_id=4b27046155b1457883bf610e3be6b70e)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:06.549328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b71b0d-44a6-4bda-9b1a-d32cb9183f66 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:06.552112 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 71/136] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:06 2022] GET /identity/v3/users/4b27046155b1457883bf610e3be6b70e => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:06.552964 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b71b0d-44a6-4bda-9b1a-d32cb9183f66 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:06.599445 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60175393-8555-4c91-9fc1-77dc0ebefc03 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:06.665113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f9080-845f-4c97-9b03-b0a32f639470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:06.666304 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f9080-845f-4c97-9b03-b0a32f639470 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:06.666864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f9080-845f-4c97-9b03-b0a32f639470 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:06.667357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f9080-845f-4c97-9b03-b0a32f639470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:06.668369 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f9080-845f-4c97-9b03-b0a32f639470 None admin] RBAC: Adding query filter params (role.id=78132e8886fd4da988a6fe763a388e16, scope.project.id=11d6ae35861149a9a30b9eadb5380b85, user.id=eebd684197654583a282fc9e4573075b) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:06.668900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f9080-845f-4c97-9b03-b0a32f639470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:06.670569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f9080-845f-4c97-9b03-b0a32f639470 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:06.677359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 66/137] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:06 2022] GET /identity/v3/role_assignments?user.id=eebd684197654583a282fc9e4573075b&scope.project.id=11d6ae35861149a9a30b9eadb5380b85&role.id=78132e8886fd4da988a6fe763a388e16 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:06.678487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f9080-845f-4c97-9b03-b0a32f639470 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:06.706241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:06.815300 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f9080-845f-4c97-9b03-b0a32f639470 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:06.828067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b71b0d-44a6-4bda-9b1a-d32cb9183f66 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:06.907659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cafa02-105b-4ee1-8d92-f199eebf1893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:06.910704 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cafa02-105b-4ee1-8d92-f199eebf1893 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:06.912268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cafa02-105b-4ee1-8d92-f199eebf1893 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:06.913289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cafa02-105b-4ee1-8d92-f199eebf1893 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:06.916051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cafa02-105b-4ee1-8d92-f199eebf1893 None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:06.918350 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a6ad1-c3f3-4e98-9937-53c7bdab189c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:06.919728 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cafa02-105b-4ee1-8d92-f199eebf1893 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:06.920599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a6ad1-c3f3-4e98-9937-53c7bdab189c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:06.921106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a6ad1-c3f3-4e98-9937-53c7bdab189c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:06.923191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 67/138] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:06 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:06.923736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a6ad1-c3f3-4e98-9937-53c7bdab189c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:06.925293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a6ad1-c3f3-4e98-9937-53c7bdab189c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:06.927097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a6ad1-c3f3-4e98-9937-53c7bdab189c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:06.929052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 72/139] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:06.934032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cafa02-105b-4ee1-8d92-f199eebf1893 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:06.938561 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a6ad1-c3f3-4e98-9937-53c7bdab189c None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:07.059349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:07.157087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a6ad1-c3f3-4e98-9937-53c7bdab189c None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:07.212645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:07.236007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6ff51-cb10-4b46-9b93-2f30e40ae901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:07.237067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6ff51-cb10-4b46-9b93-2f30e40ae901 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:07.237536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6ff51-cb10-4b46-9b93-2f30e40ae901 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:07.238026 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6ff51-cb10-4b46-9b93-2f30e40ae901 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:07.238931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6ff51-cb10-4b46-9b93-2f30e40ae901 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:07.240728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6ff51-cb10-4b46-9b93-2f30e40ae901 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:07.250681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cafa02-105b-4ee1-8d92-f199eebf1893 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:07.301219 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 73/140] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:36:06 2022] POST /identity/v3/users => generated 283 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:36:07.341570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7aa1be-0401-4aa1-8bc7-16a333e461cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:07.342275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aa1be-0401-4aa1-8bc7-16a333e461cd None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:07.342561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aa1be-0401-4aa1-8bc7-16a333e461cd None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:07.342900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7aa1be-0401-4aa1-8bc7-16a333e461cd None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85/users/4b27046155b1457883bf610e3be6b70e/roles/686764f0686e446f884315081e158bf3` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:07.348541 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7aa1be-0401-4aa1-8bc7-16a333e461cd None admin] RBAC: Authorizing `identity:create_grant(project_id=11d6ae35861149a9a30b9eadb5380b85, user_id=4b27046155b1457883bf610e3be6b70e, role_id=686764f0686e446f884315081e158bf3)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:07.350094 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7aa1be-0401-4aa1-8bc7-16a333e461cd None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:07.382520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 68/141] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:36:06 2022] PUT /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85/users/4b27046155b1457883bf610e3be6b70e/roles/686764f0686e446f884315081e158bf3 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:36:08.186584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409eee4-96d4-4eeb-9cbb-e5b858f612ed None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:08.187122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409eee4-96d4-4eeb-9cbb-e5b858f612ed None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:08.187195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409eee4-96d4-4eeb-9cbb-e5b858f612ed None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:08.188781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 74/142] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:08.194341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffe995-8f81-48c9-8968-c457294e1b10 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:08.194647 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffe995-8f81-48c9-8968-c457294e1b10 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:08.194929 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffe995-8f81-48c9-8968-c457294e1b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:08.219317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-09ffe995-8f81-48c9-8968-c457294e1b10 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:08.220500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-09ffe995-8f81-48c9-8968-c457294e1b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:08.346268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 69/143] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:08.389269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409eee4-96d4-4eeb-9cbb-e5b858f612ed None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:08.572211 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409eee4-96d4-4eeb-9cbb-e5b858f612ed None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:08.595427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:08.611698 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722590d-a14f-45c3-9d19-1d22b6b0ab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:08.613004 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722590d-a14f-45c3-9d19-1d22b6b0ab89 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:08.613528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722590d-a14f-45c3-9d19-1d22b6b0ab89 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:08.614198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722590d-a14f-45c3-9d19-1d22b6b0ab89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:08.626940 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722590d-a14f-45c3-9d19-1d22b6b0ab89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-0722590d-a14f-45c3-9d19-1d22b6b0ab89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:08.632420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:08.634375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:08.635191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:08.635191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:08.635191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:08.635191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:08.635191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:08.635191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:08.635191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:08.635191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:08.635191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:08.635191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:08.635191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:08.635191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:08.635191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 75/144] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:08.640624 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffe995-8f81-48c9-8968-c457294e1b10 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:08.707305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ed319-a55c-46ae-ac90-ba7b6b546504 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:08.707614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ed319-a55c-46ae-ac90-ba7b6b546504 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:08.707878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ed319-a55c-46ae-ac90-ba7b6b546504 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:08.709363 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 76/145] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:36:08.715262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed76d3-b642-4501-ad5f-04fe83fd7aa4 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:08.715553 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed76d3-b642-4501-ad5f-04fe83fd7aa4 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:08.715809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed76d3-b642-4501-ad5f-04fe83fd7aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:08.740407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-51ed76d3-b642-4501-ad5f-04fe83fd7aa4 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:08.741742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-51ed76d3-b642-4501-ad5f-04fe83fd7aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:08.762648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:08.829381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 77/146] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:08.831352 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffe995-8f81-48c9-8968-c457294e1b10 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:08.880261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed76d3-b642-4501-ad5f-04fe83fd7aa4 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:08.916758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d699-67b5-4745-a8e0-2f4c0bd827a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:08.917404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d699-67b5-4745-a8e0-2f4c0bd827a7 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:08.917878 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d699-67b5-4745-a8e0-2f4c0bd827a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:08.918279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d699-67b5-4745-a8e0-2f4c0bd827a7 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:08.919116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622d699-67b5-4745-a8e0-2f4c0bd827a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:08.919434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622d699-67b5-4745-a8e0-2f4c0bd827a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:08.920863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622d699-67b5-4745-a8e0-2f4c0bd827a7 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:08.929833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 70/147] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:08.952667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d699-67b5-4745-a8e0-2f4c0bd827a7 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:09.028298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed76d3-b642-4501-ad5f-04fe83fd7aa4 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:09.042902 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:09.074429 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c811c77-eefe-434d-84c6-15564e664932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:09.075129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c811c77-eefe-434d-84c6-15564e664932 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:09.075462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c811c77-eefe-434d-84c6-15564e664932 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:09.076022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c811c77-eefe-434d-84c6-15564e664932 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:09.089570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c811c77-eefe-434d-84c6-15564e664932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-3c811c77-eefe-434d-84c6-15564e664932 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:09.095431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:09.096474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:09.097247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:09.097247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:09.097247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:09.097247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:09.097247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:09.097247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:09.097247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:09.097247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:09.097247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:09.097247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:09.097247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:09.097247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:09.097247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 78/148] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:36:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:36:09.104877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c811c77-eefe-434d-84c6-15564e664932 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:09.236312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c811c77-eefe-434d-84c6-15564e664932 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:09.253804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d699-67b5-4745-a8e0-2f4c0bd827a7 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:09.277740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065b92e-d697-40da-a7eb-2688ce93b797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:09.278419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065b92e-d697-40da-a7eb-2688ce93b797 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:09.278650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065b92e-d697-40da-a7eb-2688ce93b797 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:09.278940 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065b92e-d697-40da-a7eb-2688ce93b797 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:09.280074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065b92e-d697-40da-a7eb-2688ce93b797 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:09.280395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065b92e-d697-40da-a7eb-2688ce93b797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:09.281965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065b92e-d697-40da-a7eb-2688ce93b797 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:09.294220 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 79/149] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:09.302714 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065b92e-d697-40da-a7eb-2688ce93b797 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:09.355214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:09.367414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd309d-54f7-4856-a55d-8aeff509e3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:09.368636 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd309d-54f7-4856-a55d-8aeff509e3db None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:09.369024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd309d-54f7-4856-a55d-8aeff509e3db None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:09.369310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd309d-54f7-4856-a55d-8aeff509e3db None admin] PATH_INFO: `/v3/users/4b27046155b1457883bf610e3be6b70e` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:09.370691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd309d-54f7-4856-a55d-8aeff509e3db None admin] RBAC: Authorizing `identity:get_user(user_id=4b27046155b1457883bf610e3be6b70e)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:09.372167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd309d-54f7-4856-a55d-8aeff509e3db None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:09.374010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 71/150] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:08 2022] GET /identity/v3/users/4b27046155b1457883bf610e3be6b70e => generated 275 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:09.387218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd309d-54f7-4856-a55d-8aeff509e3db None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:09.413320 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:09.431166 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065b92e-d697-40da-a7eb-2688ce93b797 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:09.470731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdd01d-8a58-48c1-b4f1-833498eea18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:09.471431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdd01d-8a58-48c1-b4f1-833498eea18b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:09.471716 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdd01d-8a58-48c1-b4f1-833498eea18b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:09.471989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdd01d-8a58-48c1-b4f1-833498eea18b None admin] PATH_INFO: `/v3/users/047b4b41672048d8889a2b3bb55b2d97` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:09.473174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bdd01d-8a58-48c1-b4f1-833498eea18b None admin] RBAC: Authorizing `identity:get_user(user_id=047b4b41672048d8889a2b3bb55b2d97)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:09.474643 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bdd01d-8a58-48c1-b4f1-833498eea18b None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:09.481604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 80/151] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:09 2022] GET /identity/v3/users/047b4b41672048d8889a2b3bb55b2d97 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:09.487162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdd01d-8a58-48c1-b4f1-833498eea18b None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:09.566985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd309d-54f7-4856-a55d-8aeff509e3db None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:09.635232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e26d82-14e9-4c30-910d-7fa8d82c5026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:09.635998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26d82-14e9-4c30-910d-7fa8d82c5026 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:09.636321 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26d82-14e9-4c30-910d-7fa8d82c5026 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:09.636601 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26d82-14e9-4c30-910d-7fa8d82c5026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:09.663000 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:09.666882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdd01d-8a58-48c1-b4f1-833498eea18b None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:09.748261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bda56-29c1-4ef3-8907-c5a371f8a218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:09.748849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bda56-29c1-4ef3-8907-c5a371f8a218 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:09.749132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bda56-29c1-4ef3-8907-c5a371f8a218 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:09.750307 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bda56-29c1-4ef3-8907-c5a371f8a218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:09.781377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e26d82-14e9-4c30-910d-7fa8d82c5026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:09.782900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e26d82-14e9-4c30-910d-7fa8d82c5026 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:09.798325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:09.887121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bda56-29c1-4ef3-8907-c5a371f8a218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:09.889033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bda56-29c1-4ef3-8907-c5a371f8a218 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:09.963094 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 72/152] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:09.964545 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d57027-dfd1-4af8-916d-c5f997c075be None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:09.964833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d57027-dfd1-4af8-916d-c5f997c075be None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:09.965109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d57027-dfd1-4af8-916d-c5f997c075be None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:09.971267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 73/153] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:09.972626 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d57027-dfd1-4af8-916d-c5f997c075be None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:10.087007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:10.091686 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 81/154] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:36:10.093375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ca97b-6c66-4bd7-9a99-9f4a95507559 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:10.095649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ca97b-6c66-4bd7-9a99-9f4a95507559 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:10.095949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ca97b-6c66-4bd7-9a99-9f4a95507559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:10.108363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d57027-dfd1-4af8-916d-c5f997c075be None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:10.126126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-2e3ca97b-6c66-4bd7-9a99-9f4a95507559 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:10.127690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2e3ca97b-6c66-4bd7-9a99-9f4a95507559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:10.150702 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:10.175432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d193998-5f96-47e8-8638-8005dd1a6756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:10.176099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d193998-5f96-47e8-8638-8005dd1a6756 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:10.176381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d193998-5f96-47e8-8638-8005dd1a6756 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:10.176651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d193998-5f96-47e8-8638-8005dd1a6756 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:10.177933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d193998-5f96-47e8-8638-8005dd1a6756 None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:10.179429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d193998-5f96-47e8-8638-8005dd1a6756 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:10.182556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 74/155] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:09 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:10.183531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d193998-5f96-47e8-8638-8005dd1a6756 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:10.235752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 82/156] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:10.236608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ca97b-6c66-4bd7-9a99-9f4a95507559 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:10.356801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d193998-5f96-47e8-8638-8005dd1a6756 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:10.396420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ca97b-6c66-4bd7-9a99-9f4a95507559 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:10.424853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1228a-0d79-4011-90c1-441d71fe8591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:10.425436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1228a-0d79-4011-90c1-441d71fe8591 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:10.425719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1228a-0d79-4011-90c1-441d71fe8591 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:10.425993 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1228a-0d79-4011-90c1-441d71fe8591 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:10.427225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1228a-0d79-4011-90c1-441d71fe8591 None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:10.428684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1228a-0d79-4011-90c1-441d71fe8591 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:10.431414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 75/157] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:10 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:10.432764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1228a-0d79-4011-90c1-441d71fe8591 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:10.452012 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:10.471499 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40ea6a-12ae-4729-b5e3-60354a604fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:10.472556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40ea6a-12ae-4729-b5e3-60354a604fe3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:10.473048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40ea6a-12ae-4729-b5e3-60354a604fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:10.473524 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40ea6a-12ae-4729-b5e3-60354a604fe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:10.484418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40ea6a-12ae-4729-b5e3-60354a604fe3 None admin] RBAC: Adding query filter params (role.id=fe6c7625134a449c8395ab84ffd82371, scope.project.id=11d6ae35861149a9a30b9eadb5380b85, user.id=4b27046155b1457883bf610e3be6b70e) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:10.485033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40ea6a-12ae-4729-b5e3-60354a604fe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:10.488526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40ea6a-12ae-4729-b5e3-60354a604fe3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:10.507273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 83/158] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:10 2022] GET /identity/v3/role_assignments?user.id=4b27046155b1457883bf610e3be6b70e&scope.project.id=11d6ae35861149a9a30b9eadb5380b85&role.id=fe6c7625134a449c8395ab84ffd82371 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:10.509463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40ea6a-12ae-4729-b5e3-60354a604fe3 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:10.604777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:10.727778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1228a-0d79-4011-90c1-441d71fe8591 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:10.776226 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40ea6a-12ae-4729-b5e3-60354a604fe3 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:10.782671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cd9f6-92e8-4452-ac0f-40deb4868616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:10.782671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cd9f6-92e8-4452-ac0f-40deb4868616 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:10.782671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cd9f6-92e8-4452-ac0f-40deb4868616 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:10.782671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cd9f6-92e8-4452-ac0f-40deb4868616 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:10.797486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cd9f6-92e8-4452-ac0f-40deb4868616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-8e0cd9f6-92e8-4452-ac0f-40deb4868616 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:10.803230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:10.804122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:10.804961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:10.804961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:10.804961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:10.804961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:10.804961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:10.804961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:10.804961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:10.804961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:10.804961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:10.804961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:10.804961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:10.804961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:10.804961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 76/159] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:10.806297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0553d7-367f-4207-931a-e56a06b165da None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:10.806583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0553d7-367f-4207-931a-e56a06b165da None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:10.807212 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0553d7-367f-4207-931a-e56a06b165da None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:10.809512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 77/160] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:10.823086 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0553d7-367f-4207-931a-e56a06b165da None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:10.866743 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbe93a-341f-4609-b258-fe18e446bc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:10.868155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe93a-341f-4609-b258-fe18e446bc99 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:10.869728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe93a-341f-4609-b258-fe18e446bc99 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:10.873161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbe93a-341f-4609-b258-fe18e446bc99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:10.874253 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbe93a-341f-4609-b258-fe18e446bc99 None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=11d6ae35861149a9a30b9eadb5380b85, user.id=047b4b41672048d8889a2b3bb55b2d97) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:10.874795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbe93a-341f-4609-b258-fe18e446bc99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:10.876563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbe93a-341f-4609-b258-fe18e446bc99 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:10.884412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 84/161] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:10 2022] GET /identity/v3/role_assignments?user.id=047b4b41672048d8889a2b3bb55b2d97&scope.project.id=11d6ae35861149a9a30b9eadb5380b85&role.id=93eb204498fc49d297bcd1841e00818d => generated 608 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:10.887060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2585c-3a4d-4481-9f24-a1febf072556 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:10.887953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2585c-3a4d-4481-9f24-a1febf072556 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:10.888674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2585c-3a4d-4481-9f24-a1febf072556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:10.927172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-afd2585c-3a4d-4481-9f24-a1febf072556 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:10.928874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-afd2585c-3a4d-4481-9f24-a1febf072556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:11.006109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0553d7-367f-4207-931a-e56a06b165da None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:11.013293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:11.047041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:11.084025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 85/162] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:11.098531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c938f0-11f0-4d2a-ac44-5c5d9f5f5718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:11.099188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c938f0-11f0-4d2a-ac44-5c5d9f5f5718 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:11.099467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c938f0-11f0-4d2a-ac44-5c5d9f5f5718 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:11.099740 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c938f0-11f0-4d2a-ac44-5c5d9f5f5718 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:11.100469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c938f0-11f0-4d2a-ac44-5c5d9f5f5718 None admin] RBAC: Adding query filter params (name=member) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:11.100771 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c938f0-11f0-4d2a-ac44-5c5d9f5f5718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:11.107359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c938f0-11f0-4d2a-ac44-5c5d9f5f5718 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:11.122603 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 78/163] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:11.131144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2585c-3a4d-4481-9f24-a1febf072556 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:11.162495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c938f0-11f0-4d2a-ac44-5c5d9f5f5718 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:11.344735 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2585c-3a4d-4481-9f24-a1febf072556 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:11.407458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c0a9c-1b23-42bd-b894-730ee5c1f024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:11.408123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c0a9c-1b23-42bd-b894-730ee5c1f024 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:11.408405 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c0a9c-1b23-42bd-b894-730ee5c1f024 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:11.408679 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c0a9c-1b23-42bd-b894-730ee5c1f024 None admin] PATH_INFO: `/v3/users/4b27046155b1457883bf610e3be6b70e` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:11.409918 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c0a9c-1b23-42bd-b894-730ee5c1f024 None admin] RBAC: Authorizing `identity:get_user(user_id=4b27046155b1457883bf610e3be6b70e)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:11.411479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c0a9c-1b23-42bd-b894-730ee5c1f024 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:11.412835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c938f0-11f0-4d2a-ac44-5c5d9f5f5718 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:11.413517 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 86/164] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:11 2022] GET /identity/v3/users/4b27046155b1457883bf610e3be6b70e => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:11.421988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c0a9c-1b23-42bd-b894-730ee5c1f024 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:11.495938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602e8ee-41fb-4eb3-839e-9a707fc88495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:11.496884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602e8ee-41fb-4eb3-839e-9a707fc88495 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:11.497177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602e8ee-41fb-4eb3-839e-9a707fc88495 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:11.497456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602e8ee-41fb-4eb3-839e-9a707fc88495 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:11.530119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602e8ee-41fb-4eb3-839e-9a707fc88495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-a602e8ee-41fb-4eb3-839e-9a707fc88495 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:11.548065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:11.549792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:11.551341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:11.551341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:11.551341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:11.551341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:11.551341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:11.551341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:11.551341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:11.551341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:11.551341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:11.551341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:11.551341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:11.551341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:11.551341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 79/165] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:11.563985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602e8ee-41fb-4eb3-839e-9a707fc88495 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:11.581187 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:11.620793 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:11.745732 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c0a9c-1b23-42bd-b894-730ee5c1f024 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:11.761856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602e8ee-41fb-4eb3-839e-9a707fc88495 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:11.815981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0ac20-295c-44c5-9915-585b71370931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:11.817043 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0ac20-295c-44c5-9915-585b71370931 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:11.817520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0ac20-295c-44c5-9915-585b71370931 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:11.818012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0ac20-295c-44c5-9915-585b71370931 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:11.819048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0ac20-295c-44c5-9915-585b71370931 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:11.819866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0ac20-295c-44c5-9915-585b71370931 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:11.821983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0ac20-295c-44c5-9915-585b71370931 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:11.828675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88ed3f-b7d7-4de2-b83c-743df12641f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:11.829661 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88ed3f-b7d7-4de2-b83c-743df12641f1 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:11.830162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88ed3f-b7d7-4de2-b83c-743df12641f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:11.830652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88ed3f-b7d7-4de2-b83c-743df12641f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:11.835757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 80/166] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:11.835757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d814c87-22c1-4a49-9c55-de0ca721e26a None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:11.838556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d814c87-22c1-4a49-9c55-de0ca721e26a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:11.838894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d814c87-22c1-4a49-9c55-de0ca721e26a None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:11.840412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 81/167] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:11.852492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d814c87-22c1-4a49-9c55-de0ca721e26a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:11.938549 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88ed3f-b7d7-4de2-b83c-743df12641f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:11.940420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88ed3f-b7d7-4de2-b83c-743df12641f1 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:12.025289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:12.076752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 87/168] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:36:12.077849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e53af-b1b4-4c10-8e45-92fa5e3e8093 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:12.078131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e53af-b1b4-4c10-8e45-92fa5e3e8093 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:12.078404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e53af-b1b4-4c10-8e45-92fa5e3e8093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:12.105184 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-113e53af-b1b4-4c10-8e45-92fa5e3e8093 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:12.106473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-113e53af-b1b4-4c10-8e45-92fa5e3e8093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:12.144921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:12.191421 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 88/169] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:12.191698 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d814c87-22c1-4a49-9c55-de0ca721e26a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:12.192631 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e53af-b1b4-4c10-8e45-92fa5e3e8093 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:12.292131 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fd0d3-c39c-4fed-bcfe-f9f247662a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:12.292847 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fd0d3-c39c-4fed-bcfe-f9f247662a4f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:12.293133 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fd0d3-c39c-4fed-bcfe-f9f247662a4f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:12.293411 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fd0d3-c39c-4fed-bcfe-f9f247662a4f None admin] PATH_INFO: `/v3/users/eebd684197654583a282fc9e4573075b` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:12.295412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fd0d3-c39c-4fed-bcfe-f9f247662a4f None admin] RBAC: Authorizing `identity:get_user(user_id=eebd684197654583a282fc9e4573075b)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:12.297186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fd0d3-c39c-4fed-bcfe-f9f247662a4f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:12.308510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 82/170] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:11 2022] GET /identity/v3/users/eebd684197654583a282fc9e4573075b => generated 289 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:12.308510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fd0d3-c39c-4fed-bcfe-f9f247662a4f None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:12.323671 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:12.356620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e53af-b1b4-4c10-8e45-92fa5e3e8093 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:12.454961 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b6022-98f7-4179-8808-e2be0e89c667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:12.455544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b6022-98f7-4179-8808-e2be0e89c667 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:12.455824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b6022-98f7-4179-8808-e2be0e89c667 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:12.456096 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b6022-98f7-4179-8808-e2be0e89c667 None admin] PATH_INFO: `/v3/projects/f13fcc67893d41a2af55704f6f81b1d4` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:12.458883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b6022-98f7-4179-8808-e2be0e89c667 None admin] RBAC: Authorizing `identity:get_project(project_id=f13fcc67893d41a2af55704f6f81b1d4)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:12.460355 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b6022-98f7-4179-8808-e2be0e89c667 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:12.462755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 89/171] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:12 2022] GET /identity/v3/projects/f13fcc67893d41a2af55704f6f81b1d4 => generated 312 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:12.463555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b6022-98f7-4179-8808-e2be0e89c667 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:12.524916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fd0d3-c39c-4fed-bcfe-f9f247662a4f None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:12.562395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012b2a9-de2b-4b46-9c29-0bbedc675717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:12.563417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012b2a9-de2b-4b46-9c29-0bbedc675717 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:12.563759 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012b2a9-de2b-4b46-9c29-0bbedc675717 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:12.564093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012b2a9-de2b-4b46-9c29-0bbedc675717 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:12.587248 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012b2a9-de2b-4b46-9c29-0bbedc675717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-a012b2a9-de2b-4b46-9c29-0bbedc675717 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:12.594941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:12.595895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:12.597005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:12.597005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:12.597005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:12.597005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:12.597005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:12.597005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:12.597005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:12.597005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:12.597005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:12.597005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:12.597005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:12.597005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:12.597005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 83/172] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:36:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:12.598469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012b2a9-de2b-4b46-9c29-0bbedc675717 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:12.620422 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b6022-98f7-4179-8808-e2be0e89c667 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:12.626293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:12.668752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79f4ba-e938-4330-a9f7-e9ed24af7685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:12.669577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79f4ba-e938-4330-a9f7-e9ed24af7685 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:12.669860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79f4ba-e938-4330-a9f7-e9ed24af7685 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:12.670186 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79f4ba-e938-4330-a9f7-e9ed24af7685 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:12.671545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79f4ba-e938-4330-a9f7-e9ed24af7685 None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:12.673060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79f4ba-e938-4330-a9f7-e9ed24af7685 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:12.675136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 90/173] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:12 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:12.676174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e020e-7c23-4e97-9443-ca8211f85f2b None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:12.676502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e020e-7c23-4e97-9443-ca8211f85f2b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:12.676771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e020e-7c23-4e97-9443-ca8211f85f2b None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:12.677986 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 91/174] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:12.678819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e020e-7c23-4e97-9443-ca8211f85f2b None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:12.707501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:12.753320 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012b2a9-de2b-4b46-9c29-0bbedc675717 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:12.797235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2014f-439d-4b6e-b666-97cce69a55c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:12.797967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2014f-439d-4b6e-b666-97cce69a55c8 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:12.798243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2014f-439d-4b6e-b666-97cce69a55c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:12.798515 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2014f-439d-4b6e-b666-97cce69a55c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:12.799510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2014f-439d-4b6e-b666-97cce69a55c8 None admin] RBAC: Adding query filter params (role.id=fe6c7625134a449c8395ab84ffd82371, scope.project.id=f13fcc67893d41a2af55704f6f81b1d4, user.id=4b27046155b1457883bf610e3be6b70e) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:12.799834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2014f-439d-4b6e-b666-97cce69a55c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:12.801350 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2014f-439d-4b6e-b666-97cce69a55c8 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:12.805376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e020e-7c23-4e97-9443-ca8211f85f2b None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:12.810308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 84/175] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:12 2022] GET /identity/v3/role_assignments?user.id=4b27046155b1457883bf610e3be6b70e&scope.project.id=f13fcc67893d41a2af55704f6f81b1d4&role.id=fe6c7625134a449c8395ab84ffd82371 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:12.811107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2014f-439d-4b6e-b666-97cce69a55c8 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:12.850686 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba18d6-49a4-48e8-a2b8-3a3f46a4e072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:12.851323 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba18d6-49a4-48e8-a2b8-3a3f46a4e072 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:12.851594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba18d6-49a4-48e8-a2b8-3a3f46a4e072 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:12.852384 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba18d6-49a4-48e8-a2b8-3a3f46a4e072 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:12.853127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba18d6-49a4-48e8-a2b8-3a3f46a4e072 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:12.853427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba18d6-49a4-48e8-a2b8-3a3f46a4e072 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:12.855467 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba18d6-49a4-48e8-a2b8-3a3f46a4e072 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:12.870860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 92/176] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:36:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:12.872279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4031838-445e-4fae-9606-c57f4becaf67 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:12.872556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4031838-445e-4fae-9606-c57f4becaf67 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:12.872828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4031838-445e-4fae-9606-c57f4becaf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:12.900601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c4031838-445e-4fae-9606-c57f4becaf67 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:12.902284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c4031838-445e-4fae-9606-c57f4becaf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:12.942585 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:12.971319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2014f-439d-4b6e-b666-97cce69a55c8 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:13.006919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 93/177] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:13.009677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144464a-1d6b-4c3c-97fa-3e37fa5570dd None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:13.010132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144464a-1d6b-4c3c-97fa-3e37fa5570dd None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:13.010571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144464a-1d6b-4c3c-97fa-3e37fa5570dd None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:13.014752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 94/178] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:13.016155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144464a-1d6b-4c3c-97fa-3e37fa5570dd None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:13.017501 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844fff0-29be-4812-9369-0a0dbd2ebf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:13.018731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844fff0-29be-4812-9369-0a0dbd2ebf25 None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:13.019183 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844fff0-29be-4812-9369-0a0dbd2ebf25 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:13.019626 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844fff0-29be-4812-9369-0a0dbd2ebf25 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85/users/eebd684197654583a282fc9e4573075b/roles/78132e8886fd4da988a6fe763a388e16` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:13.022574 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844fff0-29be-4812-9369-0a0dbd2ebf25 None admin] RBAC: Authorizing `identity:create_grant(project_id=11d6ae35861149a9a30b9eadb5380b85, user_id=eebd684197654583a282fc9e4573075b, role_id=78132e8886fd4da988a6fe763a388e16)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:13.026354 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844fff0-29be-4812-9369-0a0dbd2ebf25 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:13.043231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:13.048572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 85/179] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:36:12 2022] PUT /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85/users/eebd684197654583a282fc9e4573075b/roles/78132e8886fd4da988a6fe763a388e16 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:36:13.050177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c58a39-ce3e-44e5-a58f-87e82bc1fb56 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:13.050537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c58a39-ce3e-44e5-a58f-87e82bc1fb56 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:13.050917 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c58a39-ce3e-44e5-a58f-87e82bc1fb56 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:13.052941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 86/180] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:13.054671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b5181-5fe4-4e95-a442-5d7424a7ae7c None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:13.055206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b5181-5fe4-4e95-a442-5d7424a7ae7c None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:13.055684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b5181-5fe4-4e95-a442-5d7424a7ae7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:13.084499 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-a72b5181-5fe4-4e95-a442-5d7424a7ae7c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:13.087073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a72b5181-5fe4-4e95-a442-5d7424a7ae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:13.200136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 87/181] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:13.201026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b5181-5fe4-4e95-a442-5d7424a7ae7c None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:13.203882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144464a-1d6b-4c3c-97fa-3e37fa5570dd None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:13.261433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048baaa7-46c6-4355-bc4d-b7d9bb13a92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:13.262122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048baaa7-46c6-4355-bc4d-b7d9bb13a92a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:13.262406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048baaa7-46c6-4355-bc4d-b7d9bb13a92a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:13.262692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048baaa7-46c6-4355-bc4d-b7d9bb13a92a None admin] PATH_INFO: `/v3/users/4b27046155b1457883bf610e3be6b70e` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:13.264533 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048baaa7-46c6-4355-bc4d-b7d9bb13a92a None admin] RBAC: Authorizing `identity:get_user(user_id=4b27046155b1457883bf610e3be6b70e)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:13.267151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048baaa7-46c6-4355-bc4d-b7d9bb13a92a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:13.269239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 95/182] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:13 2022] GET /identity/v3/users/4b27046155b1457883bf610e3be6b70e => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:13.270250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def32ab8-f70b-4a84-9b27-41cb80421694 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:13.270529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def32ab8-f70b-4a84-9b27-41cb80421694 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:13.270840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def32ab8-f70b-4a84-9b27-41cb80421694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:13.293206 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-def32ab8-f70b-4a84-9b27-41cb80421694 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:13.294566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-def32ab8-f70b-4a84-9b27-41cb80421694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:13.315412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:13.366001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b5181-5fe4-4e95-a442-5d7424a7ae7c None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:13.395363 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 96/183] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:13.396388 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def32ab8-f70b-4a84-9b27-41cb80421694 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:13.411964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d041e-4202-4e21-a173-c9756ef8bc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:13.412967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d041e-4202-4e21-a173-c9756ef8bc8c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:13.413446 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d041e-4202-4e21-a173-c9756ef8bc8c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:13.413964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d041e-4202-4e21-a173-c9756ef8bc8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:13.419563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:13.427081 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d041e-4202-4e21-a173-c9756ef8bc8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-548d041e-4202-4e21-a173-c9756ef8bc8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:13.434617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:13.436509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:13.437426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:13.437426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:13.437426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:13.437426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:13.437426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:13.437426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:13.437426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:13.437426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:13.437426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:13.437426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:13.437426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:13.437426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:13.437426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 88/184] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:36:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:13.438432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d041e-4202-4e21-a173-c9756ef8bc8c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:13.551654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def32ab8-f70b-4a84-9b27-41cb80421694 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:13.573672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d041e-4202-4e21-a173-c9756ef8bc8c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:13.601364 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa2467-75f0-4556-8936-cc06e87916a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:13.603092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa2467-75f0-4556-8936-cc06e87916a7 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:13.603553 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa2467-75f0-4556-8936-cc06e87916a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:13.603970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa2467-75f0-4556-8936-cc06e87916a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:13.605348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa2467-75f0-4556-8936-cc06e87916a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:13.606843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa2467-75f0-4556-8936-cc06e87916a7 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:13.608678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 97/185] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:13.612322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:13.614250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa2467-75f0-4556-8936-cc06e87916a7 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:13.621406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8531558-cd1e-424c-9fe9-79e8c77d6b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:13.622175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8531558-cd1e-424c-9fe9-79e8c77d6b15 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:13.622448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8531558-cd1e-424c-9fe9-79e8c77d6b15 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:13.622743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8531558-cd1e-424c-9fe9-79e8c77d6b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:13.712249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:13.716551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8531558-cd1e-424c-9fe9-79e8c77d6b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:13.718507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8531558-cd1e-424c-9fe9-79e8c77d6b15 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:13.734031 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa2467-75f0-4556-8936-cc06e87916a7 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:13.781330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814ebd6-ff17-4eea-a122-c4ee0cc069e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:13.782159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814ebd6-ff17-4eea-a122-c4ee0cc069e0 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:13.782576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814ebd6-ff17-4eea-a122-c4ee0cc069e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:13.782994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814ebd6-ff17-4eea-a122-c4ee0cc069e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:13.797437 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814ebd6-ff17-4eea-a122-c4ee0cc069e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-e814ebd6-ff17-4eea-a122-c4ee0cc069e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:13.802979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:13.803903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:13.804842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:13.804842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:13.804842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:13.804842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:13.804842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:13.804842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:13.804842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:13.804842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:13.804842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:13.804842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:13.804842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:13.804842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:13.804842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 98/186] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:36:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:13.811254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814ebd6-ff17-4eea-a122-c4ee0cc069e0 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:13.849249 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 89/187] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:13.850810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8531558-cd1e-424c-9fe9-79e8c77d6b15 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:13.887134 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:13.946164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814ebd6-ff17-4eea-a122-c4ee0cc069e0 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:13.982228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8531558-cd1e-424c-9fe9-79e8c77d6b15 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:13.990663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5e794-a731-4b52-a6dd-b229ff118188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:13.991285 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e794-a731-4b52-a6dd-b229ff118188 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:13.991565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e794-a731-4b52-a6dd-b229ff118188 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:13.991832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e794-a731-4b52-a6dd-b229ff118188 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:13.993473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5e794-a731-4b52-a6dd-b229ff118188 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:13.993781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5e794-a731-4b52-a6dd-b229ff118188 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:13.995842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5e794-a731-4b52-a6dd-b229ff118188 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:14.005568 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 99/188] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:14.012548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5e794-a731-4b52-a6dd-b229ff118188 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:14.016652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:14.022779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1015065-cf8b-4650-8df5-02647e4c6e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:14.023399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1015065-cf8b-4650-8df5-02647e4c6e5d None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:14.023714 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1015065-cf8b-4650-8df5-02647e4c6e5d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:14.024704 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1015065-cf8b-4650-8df5-02647e4c6e5d None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:14.025400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1015065-cf8b-4650-8df5-02647e4c6e5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:14.027191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1015065-cf8b-4650-8df5-02647e4c6e5d None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:14.066739 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 90/189] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:36:13 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:36:14.074440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1015065-cf8b-4650-8df5-02647e4c6e5d None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:14.171618 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5e794-a731-4b52-a6dd-b229ff118188 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:14.211964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:14.234120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a39e0-3213-4601-b386-92be5c7141e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:14.234791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a39e0-3213-4601-b386-92be5c7141e6 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:14.235075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a39e0-3213-4601-b386-92be5c7141e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:14.235571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a39e0-3213-4601-b386-92be5c7141e6 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:14.236311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a39e0-3213-4601-b386-92be5c7141e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:14.236619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a39e0-3213-4601-b386-92be5c7141e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:14.238052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a39e0-3213-4601-b386-92be5c7141e6 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:14.238489 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1015065-cf8b-4650-8df5-02647e4c6e5d None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:14.249741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 100/190] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:36:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:14.250561 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a39e0-3213-4601-b386-92be5c7141e6 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:14.284244 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01d69e-9039-421c-9130-c848c4f856ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:14.284813 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01d69e-9039-421c-9130-c848c4f856ba None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:14.285774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01d69e-9039-421c-9130-c848c4f856ba None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:14.285774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01d69e-9039-421c-9130-c848c4f856ba None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:14.287462 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01d69e-9039-421c-9130-c848c4f856ba None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:14.290146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01d69e-9039-421c-9130-c848c4f856ba None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:14.293401 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 91/191] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:14 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:14.294980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01d69e-9039-421c-9130-c848c4f856ba None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:14.357215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:14.406045 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a39e0-3213-4601-b386-92be5c7141e6 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:14.443256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01d69e-9039-421c-9130-c848c4f856ba None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:14.457098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed162a-465e-4800-9661-22f00dcbf272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:14.457753 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed162a-465e-4800-9661-22f00dcbf272 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:14.458033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed162a-465e-4800-9661-22f00dcbf272 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:14.458306 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed162a-465e-4800-9661-22f00dcbf272 None admin] PATH_INFO: `/v3/users/db884278393f4de19359c6c1e560f1bd` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:14.482368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed162a-465e-4800-9661-22f00dcbf272 None admin] RBAC: Authorizing `identity:get_user(user_id=db884278393f4de19359c6c1e560f1bd)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:14.484001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed162a-465e-4800-9661-22f00dcbf272 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:14.486480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 101/192] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:14 2022] GET /identity/v3/users/db884278393f4de19359c6c1e560f1bd => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:14.487914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed162a-465e-4800-9661-22f00dcbf272 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:14.501264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad432104-f7a3-46f4-948e-65c37fede1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:14.501912 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad432104-f7a3-46f4-948e-65c37fede1e4 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:14.502183 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad432104-f7a3-46f4-948e-65c37fede1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:14.502446 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad432104-f7a3-46f4-948e-65c37fede1e4 None admin] PATH_INFO: `/v3/users/db884278393f4de19359c6c1e560f1bd` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:14.503691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad432104-f7a3-46f4-948e-65c37fede1e4 None admin] RBAC: Authorizing `identity:get_user(user_id=db884278393f4de19359c6c1e560f1bd)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:14.505177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad432104-f7a3-46f4-948e-65c37fede1e4 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:14.510972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 92/193] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:14 2022] GET /identity/v3/users/db884278393f4de19359c6c1e560f1bd => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:14.510972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad432104-f7a3-46f4-948e-65c37fede1e4 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:14.546801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:14.647974 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed162a-465e-4800-9661-22f00dcbf272 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:14.668714 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad432104-f7a3-46f4-948e-65c37fede1e4 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:14.683042 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:14.708693 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb747790-1ba3-466c-a592-d1f289de867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:14.709319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb747790-1ba3-466c-a592-d1f289de867e None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:14.709592 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb747790-1ba3-466c-a592-d1f289de867e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:14.709861 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb747790-1ba3-466c-a592-d1f289de867e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:14.710703 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb747790-1ba3-466c-a592-d1f289de867e None admin] RBAC: Adding query filter params (role.id=686764f0686e446f884315081e158bf3, scope.project.id=11d6ae35861149a9a30b9eadb5380b85, user.id=4b27046155b1457883bf610e3be6b70e) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:14.711046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb747790-1ba3-466c-a592-d1f289de867e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:14.712717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb747790-1ba3-466c-a592-d1f289de867e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:14.729986 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1159e3-f1b6-44a9-85d9-a3c9dfef6ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:14.730638 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1159e3-f1b6-44a9-85d9-a3c9dfef6ef9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:14.735202 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 102/194] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:14 2022] GET /identity/v3/role_assignments?user.id=4b27046155b1457883bf610e3be6b70e&scope.project.id=11d6ae35861149a9a30b9eadb5380b85&role.id=686764f0686e446f884315081e158bf3 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:14.735678 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1159e3-f1b6-44a9-85d9-a3c9dfef6ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:14.736037 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1159e3-f1b6-44a9-85d9-a3c9dfef6ef9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:14.737580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb747790-1ba3-466c-a592-d1f289de867e None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:14.912189 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1159e3-f1b6-44a9-85d9-a3c9dfef6ef9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:14.913604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1159e3-f1b6-44a9-85d9-a3c9dfef6ef9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:14.948628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:15.040663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb747790-1ba3-466c-a592-d1f289de867e None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:15.085432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 93/195] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:15.093463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1159e3-f1b6-44a9-85d9-a3c9dfef6ef9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:15.121583 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03026905-a5f6-4607-94ad-3c783eac9db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:15.122349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03026905-a5f6-4607-94ad-3c783eac9db1 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:15.122485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03026905-a5f6-4607-94ad-3c783eac9db1 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:15.122781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03026905-a5f6-4607-94ad-3c783eac9db1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:15.213745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:15.230297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03026905-a5f6-4607-94ad-3c783eac9db1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:15.232031 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03026905-a5f6-4607-94ad-3c783eac9db1 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:15.333296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1159e3-f1b6-44a9-85d9-a3c9dfef6ef9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:15.352784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 103/196] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:15.362697 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03026905-a5f6-4607-94ad-3c783eac9db1 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:15.379799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:15.411505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05344b72-d611-419b-8b4d-b37bf74c62e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:15.412131 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05344b72-d611-419b-8b4d-b37bf74c62e8 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:15.412439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05344b72-d611-419b-8b4d-b37bf74c62e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:15.412712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05344b72-d611-419b-8b4d-b37bf74c62e8 None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:15.415775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05344b72-d611-419b-8b4d-b37bf74c62e8 None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:15.417279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05344b72-d611-419b-8b4d-b37bf74c62e8 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:15.419098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 94/197] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:15 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:15.425168 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05344b72-d611-419b-8b4d-b37bf74c62e8 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:15.479361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:15.502846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03026905-a5f6-4607-94ad-3c783eac9db1 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:15.552937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b3207-9e2c-46ea-9700-eab6a3661bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:15.553848 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b3207-9e2c-46ea-9700-eab6a3661bc7 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:15.554340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b3207-9e2c-46ea-9700-eab6a3661bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:15.555239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b3207-9e2c-46ea-9700-eab6a3661bc7 None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:15.556953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b3207-9e2c-46ea-9700-eab6a3661bc7 None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:15.558896 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b3207-9e2c-46ea-9700-eab6a3661bc7 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:15.560869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 104/198] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:15 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:15.562338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0ecd9-27ed-4c2e-bf0b-c5e5d1cd8a0b None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:15.562809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0ecd9-27ed-4c2e-bf0b-c5e5d1cd8a0b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:15.563461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0ecd9-27ed-4c2e-bf0b-c5e5d1cd8a0b None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:15.564899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 105/199] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:15.570983 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cc5bf-0e1e-48b3-8d75-4180c71ea2b8 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:15.571700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cc5bf-0e1e-48b3-8d75-4180c71ea2b8 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:15.572175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cc5bf-0e1e-48b3-8d75-4180c71ea2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:15.575291 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05344b72-d611-419b-8b4d-b37bf74c62e8 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:15.592371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-595cc5bf-0e1e-48b3-8d75-4180c71ea2b8 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:15.593906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-595cc5bf-0e1e-48b3-8d75-4180c71ea2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:15.614242 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669876a-9997-47e5-a79a-49ea9942956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:15.615185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669876a-9997-47e5-a79a-49ea9942956c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:15.615356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669876a-9997-47e5-a79a-49ea9942956c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:15.615534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669876a-9997-47e5-a79a-49ea9942956c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:15.616341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669876a-9997-47e5-a79a-49ea9942956c None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=5c33c6fa415942299549b8fdce48c1b7, user.id=db884278393f4de19359c6c1e560f1bd) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:15.616649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669876a-9997-47e5-a79a-49ea9942956c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:15.618749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669876a-9997-47e5-a79a-49ea9942956c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:15.631078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 95/200] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:15 2022] GET /identity/v3/role_assignments?user.id=db884278393f4de19359c6c1e560f1bd&scope.project.id=5c33c6fa415942299549b8fdce48c1b7&role.id=93eb204498fc49d297bcd1841e00818d => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:15.631983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669876a-9997-47e5-a79a-49ea9942956c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:15.687801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 106/201] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:15.713765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cc5bf-0e1e-48b3-8d75-4180c71ea2b8 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:15.744042 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:15.819060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:15.839742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cc5bf-0e1e-48b3-8d75-4180c71ea2b8 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:15.856096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669876a-9997-47e5-a79a-49ea9942956c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:15.880900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9679026-2ac0-4bc4-935a-43e873dade64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:15.881839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9679026-2ac0-4bc4-935a-43e873dade64 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:15.882258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9679026-2ac0-4bc4-935a-43e873dade64 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:15.882699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9679026-2ac0-4bc4-935a-43e873dade64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:15.895554 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9679026-2ac0-4bc4-935a-43e873dade64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-b9679026-2ac0-4bc4-935a-43e873dade64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:15.901239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:15.902776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:15.904585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:15.904585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:15.904585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:15.904585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:15.904585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:15.904585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:15.904585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:15.904585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:15.904585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:15.904585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:15.904585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:15.904585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:15.907347 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 107/202] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:15.914662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9679026-2ac0-4bc4-935a-43e873dade64 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:15.974618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4780aa-6f0e-405c-8f2d-7fbabfe50906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:15.975543 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4780aa-6f0e-405c-8f2d-7fbabfe50906 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:15.975884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4780aa-6f0e-405c-8f2d-7fbabfe50906 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:15.976227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4780aa-6f0e-405c-8f2d-7fbabfe50906 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:15.977862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4780aa-6f0e-405c-8f2d-7fbabfe50906 None admin] RBAC: Adding query filter params (role.id=686764f0686e446f884315081e158bf3, scope.project.id=5c33c6fa415942299549b8fdce48c1b7, user.id=db884278393f4de19359c6c1e560f1bd) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:15.977862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4780aa-6f0e-405c-8f2d-7fbabfe50906 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:15.984318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4780aa-6f0e-405c-8f2d-7fbabfe50906 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:16.004687 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 96/203] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:15 2022] GET /identity/v3/role_assignments?user.id=db884278393f4de19359c6c1e560f1bd&scope.project.id=5c33c6fa415942299549b8fdce48c1b7&role.id=686764f0686e446f884315081e158bf3 => generated 259 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:16.053241 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9679026-2ac0-4bc4-935a-43e873dade64 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:16.100622 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16863841-af86-4b49-8ca1-b904883ca932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:16.101524 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16863841-af86-4b49-8ca1-b904883ca932 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:16.103378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16863841-af86-4b49-8ca1-b904883ca932 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:16.104080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16863841-af86-4b49-8ca1-b904883ca932 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:16.105529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16863841-af86-4b49-8ca1-b904883ca932 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:16.106105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16863841-af86-4b49-8ca1-b904883ca932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:16.107770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16863841-af86-4b49-8ca1-b904883ca932 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:16.123478 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 108/204] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:16.129502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4780aa-6f0e-405c-8f2d-7fbabfe50906 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:16.319891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4780aa-6f0e-405c-8f2d-7fbabfe50906 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:16.360619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:16.380952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa9954-e7b3-43bb-b1bf-403176e25403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:16.381674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa9954-e7b3-43bb-b1bf-403176e25403 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:16.381954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa9954-e7b3-43bb-b1bf-403176e25403 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:16.382227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa9954-e7b3-43bb-b1bf-403176e25403 None admin] PATH_INFO: `/v3/users/eebd684197654583a282fc9e4573075b` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:16.384286 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa9954-e7b3-43bb-b1bf-403176e25403 None admin] RBAC: Authorizing `identity:get_user(user_id=eebd684197654583a282fc9e4573075b)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:16.385759 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa9954-e7b3-43bb-b1bf-403176e25403 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:16.387439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 97/205] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:16 2022] GET /identity/v3/users/eebd684197654583a282fc9e4573075b => generated 289 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:16.393089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16863841-af86-4b49-8ca1-b904883ca932 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:16.404148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:16.517495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16863841-af86-4b49-8ca1-b904883ca932 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:16.561048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65687a-bbd9-4cf8-bf30-a5b30e8a28e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:16.561736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65687a-bbd9-4cf8-bf30-a5b30e8a28e5 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:16.562009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65687a-bbd9-4cf8-bf30-a5b30e8a28e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:16.562280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65687a-bbd9-4cf8-bf30-a5b30e8a28e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:16.654137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65687a-bbd9-4cf8-bf30-a5b30e8a28e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:16.655732 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65687a-bbd9-4cf8-bf30-a5b30e8a28e5 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:16.665198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:16.787494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 109/206] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:36:16.794347 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa9954-e7b3-43bb-b1bf-403176e25403 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:16.928634 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa9954-e7b3-43bb-b1bf-403176e25403 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:16.975983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f92f71-e119-43fe-8ed3-df079b619ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:16.976670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f92f71-e119-43fe-8ed3-df079b619ef3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:16.976981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f92f71-e119-43fe-8ed3-df079b619ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:16.977232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f92f71-e119-43fe-8ed3-df079b619ef3 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:16.978449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f92f71-e119-43fe-8ed3-df079b619ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:16.979932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f92f71-e119-43fe-8ed3-df079b619ef3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:16.981950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 98/207] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:16 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:16.988025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65687a-bbd9-4cf8-bf30-a5b30e8a28e5 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:17.144965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65687a-bbd9-4cf8-bf30-a5b30e8a28e5 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:17.154954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:17.192807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba607859-39ea-4559-9cd8-dad8df19bdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:17.193361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba607859-39ea-4559-9cd8-dad8df19bdd5 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:17.193641 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba607859-39ea-4559-9cd8-dad8df19bdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:17.193912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba607859-39ea-4559-9cd8-dad8df19bdd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:17.194746 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba607859-39ea-4559-9cd8-dad8df19bdd5 None admin] RBAC: Adding query filter params (role.id=78132e8886fd4da988a6fe763a388e16, scope.project.id=11d6ae35861149a9a30b9eadb5380b85, user.id=eebd684197654583a282fc9e4573075b) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:17.195059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba607859-39ea-4559-9cd8-dad8df19bdd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:17.197611 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba607859-39ea-4559-9cd8-dad8df19bdd5 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:17.200165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf79fad-7442-4e30-87f6-e2ec272143d1 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:17.200453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf79fad-7442-4e30-87f6-e2ec272143d1 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:17.200725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf79fad-7442-4e30-87f6-e2ec272143d1 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:17.202283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 99/208] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:17.205036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 110/209] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:16 2022] GET /identity/v3/role_assignments?user.id=eebd684197654583a282fc9e4573075b&scope.project.id=11d6ae35861149a9a30b9eadb5380b85&role.id=78132e8886fd4da988a6fe763a388e16 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:17.221938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5dca42-9215-4ed3-881f-d3dcf6daf124 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:17.222324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5dca42-9215-4ed3-881f-d3dcf6daf124 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:17.222466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5dca42-9215-4ed3-881f-d3dcf6daf124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:17.253534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-dc5dca42-9215-4ed3-881f-d3dcf6daf124 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:17.259037 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-dc5dca42-9215-4ed3-881f-d3dcf6daf124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:17.355663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:17.388070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 100/210] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:17.412256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba607859-39ea-4559-9cd8-dad8df19bdd5 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:17.540744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba607859-39ea-4559-9cd8-dad8df19bdd5 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:17.586156 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953baa2-f27f-46a8-840b-329d5bffb305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:17.587286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953baa2-f27f-46a8-840b-329d5bffb305 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:17.587594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953baa2-f27f-46a8-840b-329d5bffb305 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:17.587907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953baa2-f27f-46a8-840b-329d5bffb305 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:17.599637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953baa2-f27f-46a8-840b-329d5bffb305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-b953baa2-f27f-46a8-840b-329d5bffb305 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:17.606267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:17.607214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:17.607999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:17.607999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:17.607999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:17.607999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:17.607999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:17.607999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:17.607999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:17.607999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:17.607999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:17.607999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:17.607999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:17.607999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:17.607999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 111/211] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:17.613707 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5dca42-9215-4ed3-881f-d3dcf6daf124 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:17.748947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5dca42-9215-4ed3-881f-d3dcf6daf124 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:17.797123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3229c-d8e1-4715-9696-4a84accf89ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:17.798613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c0cf3-4765-467d-b677-1342d273a71f None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:17.798963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c0cf3-4765-467d-b677-1342d273a71f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:17.799168 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3229c-d8e1-4715-9696-4a84accf89ce None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:17.799322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c0cf3-4765-467d-b677-1342d273a71f None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:17.799813 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3229c-d8e1-4715-9696-4a84accf89ce None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:17.800281 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3229c-d8e1-4715-9696-4a84accf89ce None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:17.800701 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 112/212] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:36:17.801610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3229c-d8e1-4715-9696-4a84accf89ce None admin] RBAC: Adding query filter params (name=member) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:17.802113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3229c-d8e1-4715-9696-4a84accf89ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:17.803739 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3229c-d8e1-4715-9696-4a84accf89ce None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:17.812630 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aabfb-aede-4a9d-ac74-7b2ea1ed0b17 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:17.813487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aabfb-aede-4a9d-ac74-7b2ea1ed0b17 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:17.813977 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aabfb-aede-4a9d-ac74-7b2ea1ed0b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:17.820490 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 101/213] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:17.833148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3229c-d8e1-4715-9696-4a84accf89ce None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:17.854771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-cf9aabfb-aede-4a9d-ac74-7b2ea1ed0b17 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:17.856081 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-cf9aabfb-aede-4a9d-ac74-7b2ea1ed0b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:17.889431 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:17.896743 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:17.939282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 113/214] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:17.963818 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3229c-d8e1-4715-9696-4a84accf89ce None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:17.970962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9aabfb-aede-4a9d-ac74-7b2ea1ed0b17 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:18.002986 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9799a01-87bf-4602-8104-1e7e04e100bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:18.003556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9799a01-87bf-4602-8104-1e7e04e100bc None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:18.003833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9799a01-87bf-4602-8104-1e7e04e100bc None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:18.004100 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9799a01-87bf-4602-8104-1e7e04e100bc None admin] PATH_INFO: `/v3/users/8b0baa8bb10844e4bde9c20839ed5733` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:18.024504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9799a01-87bf-4602-8104-1e7e04e100bc None admin] RBAC: Authorizing `identity:get_user(user_id=8b0baa8bb10844e4bde9c20839ed5733)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:18.026417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9799a01-87bf-4602-8104-1e7e04e100bc None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:18.028796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 102/215] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:17 2022] GET /identity/v3/users/8b0baa8bb10844e4bde9c20839ed5733 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:18.034609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9799a01-87bf-4602-8104-1e7e04e100bc None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:18.101150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9aabfb-aede-4a9d-ac74-7b2ea1ed0b17 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:18.145893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb11973-9d10-4b75-bce3-ac8c48830eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:18.146994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb11973-9d10-4b75-bce3-ac8c48830eff None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:18.147469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb11973-9d10-4b75-bce3-ac8c48830eff None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:18.147930 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb11973-9d10-4b75-bce3-ac8c48830eff None admin] PATH_INFO: `/v3/domains/default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:18.149905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb11973-9d10-4b75-bce3-ac8c48830eff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:18.152097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb11973-9d10-4b75-bce3-ac8c48830eff None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:18.155107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 114/216] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:18.166174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb11973-9d10-4b75-bce3-ac8c48830eff None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:18.166357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9799a01-87bf-4602-8104-1e7e04e100bc None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:18.171750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:18.210782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:18.225301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d51e82-c1da-4abe-9385-88253d24dfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:18.226396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d51e82-c1da-4abe-9385-88253d24dfb3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:18.227940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d51e82-c1da-4abe-9385-88253d24dfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:18.228466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d51e82-c1da-4abe-9385-88253d24dfb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:18.308161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb11973-9d10-4b75-bce3-ac8c48830eff None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:18.339979 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d51e82-c1da-4abe-9385-88253d24dfb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:18.342578 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d51e82-c1da-4abe-9385-88253d24dfb3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:18.352933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef404fc1-a610-4a71-a489-7f89835c5745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:18.359873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef404fc1-a610-4a71-a489-7f89835c5745 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:18.359873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef404fc1-a610-4a71-a489-7f89835c5745 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:18.359873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef404fc1-a610-4a71-a489-7f89835c5745 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:18.360366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef404fc1-a610-4a71-a489-7f89835c5745 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:18.361862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef404fc1-a610-4a71-a489-7f89835c5745 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:18.410942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 115/217] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:36:18 2022] POST /identity/v3/users => generated 297 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:36:18.524304 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 103/218] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:18.532075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef404fc1-a610-4a71-a489-7f89835c5745 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:18.566152 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a6d30-20da-4275-a208-5561c389d1e1 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:18.566335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a6d30-20da-4275-a208-5561c389d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:18.566610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a6d30-20da-4275-a208-5561c389d1e1 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:18.568136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 104/219] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:36:18.574949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcaed6-7ad5-4e41-b3b4-0e60e9f12f74 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:18.575336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcaed6-7ad5-4e41-b3b4-0e60e9f12f74 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:18.575610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcaed6-7ad5-4e41-b3b4-0e60e9f12f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:18.583397 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:18.598204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-36bcaed6-7ad5-4e41-b3b4-0e60e9f12f74 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:18.599432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-36bcaed6-7ad5-4e41-b3b4-0e60e9f12f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:18.650936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:18.677789 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef404fc1-a610-4a71-a489-7f89835c5745 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:18.718666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 105/220] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:18.719724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423b5f7-a227-4215-a8dd-0d7cc7484ab5 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:18.719724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423b5f7-a227-4215-a8dd-0d7cc7484ab5 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:18.719989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423b5f7-a227-4215-a8dd-0d7cc7484ab5 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:18.721380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 106/221] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:18.728289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f92e85-fa48-4259-9f29-9a782b89b6ee None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:18.728573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f92e85-fa48-4259-9f29-9a782b89b6ee None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:18.728846 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f92e85-fa48-4259-9f29-9a782b89b6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:18.735336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824328f-694d-4ea1-8989-57433643f215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:18.735991 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824328f-694d-4ea1-8989-57433643f215 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:18.736268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824328f-694d-4ea1-8989-57433643f215 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:18.736541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824328f-694d-4ea1-8989-57433643f215 None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:18.738874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824328f-694d-4ea1-8989-57433643f215 None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:18.740353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824328f-694d-4ea1-8989-57433643f215 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:18.743667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 116/222] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:18 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:18.744817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824328f-694d-4ea1-8989-57433643f215 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:18.759899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-30f92e85-fa48-4259-9f29-9a782b89b6ee None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:18.761122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-30f92e85-fa48-4259-9f29-9a782b89b6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:18.877269 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 107/223] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:18.877862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f92e85-fa48-4259-9f29-9a782b89b6ee None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:18.886034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824328f-694d-4ea1-8989-57433643f215 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:18.947695 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a25f21-eec9-4596-95b1-fe9c902a7080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:18.948354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a25f21-eec9-4596-95b1-fe9c902a7080 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:18.948638 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a25f21-eec9-4596-95b1-fe9c902a7080 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:18.948908 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a25f21-eec9-4596-95b1-fe9c902a7080 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:18.964722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a25f21-eec9-4596-95b1-fe9c902a7080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:18.967622 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-72a25f21-eec9-4596-95b1-fe9c902a7080 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:18.974395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:18.975617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:18.976494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:18.976494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:18.976494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:18.976494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:18.976494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:18.976494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:18.976494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:18.976494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:18.976494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:18.976494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:18.976494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:18.976494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:18.976494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 117/224] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:36:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:18.977381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a25f21-eec9-4596-95b1-fe9c902a7080 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:18.994574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:19.019375 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f92e85-fa48-4259-9f29-9a782b89b6ee None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:19.059765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1e7b8-77bd-44ed-9996-507c01e956ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:19.060568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1e7b8-77bd-44ed-9996-507c01e956ae None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:19.060963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1e7b8-77bd-44ed-9996-507c01e956ae None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:19.061336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1e7b8-77bd-44ed-9996-507c01e956ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:19.062383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1e7b8-77bd-44ed-9996-507c01e956ae None admin] RBAC: Adding query filter params (role.id=fe6c7625134a449c8395ab84ffd82371, scope.project.id=5c33c6fa415942299549b8fdce48c1b7, user.id=8b0baa8bb10844e4bde9c20839ed5733) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:19.063094 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1e7b8-77bd-44ed-9996-507c01e956ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:19.065178 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1e7b8-77bd-44ed-9996-507c01e956ae None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:19.073851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 108/225] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:18 2022] GET /identity/v3/role_assignments?user.id=8b0baa8bb10844e4bde9c20839ed5733&scope.project.id=5c33c6fa415942299549b8fdce48c1b7&role.id=fe6c7625134a449c8395ab84ffd82371 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:19.075051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1e7b8-77bd-44ed-9996-507c01e956ae None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:19.107063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a25f21-eec9-4596-95b1-fe9c902a7080 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:19.154447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb5ace-564b-47a5-8c22-855ca8f768f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:19.155051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb5ace-564b-47a5-8c22-855ca8f768f9 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:19.155431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb5ace-564b-47a5-8c22-855ca8f768f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:19.155702 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb5ace-564b-47a5-8c22-855ca8f768f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:19.171341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cb5ace-564b-47a5-8c22-855ca8f768f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-95cb5ace-564b-47a5-8c22-855ca8f768f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:19.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:19.178127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:19.180505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:19.180505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:19.180505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:19.180505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:19.180505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:19.180505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:19.180505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:19.180505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:19.180505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:19.180505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:19.180505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:19.180505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:19.180505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 118/226] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:36:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:19.187393 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb5ace-564b-47a5-8c22-855ca8f768f9 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:19.214456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1e7b8-77bd-44ed-9996-507c01e956ae None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:19.246878 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:19.256742 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c5d3a-50d4-4c16-9d5e-43cc12b6f162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:19.257544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c5d3a-50d4-4c16-9d5e-43cc12b6f162 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:19.258180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c5d3a-50d4-4c16-9d5e-43cc12b6f162 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:19.258479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c5d3a-50d4-4c16-9d5e-43cc12b6f162 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:19.259558 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c5d3a-50d4-4c16-9d5e-43cc12b6f162 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:19.260166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c5d3a-50d4-4c16-9d5e-43cc12b6f162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:19.262448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c5d3a-50d4-4c16-9d5e-43cc12b6f162 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:19.275457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 109/227] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:19.279812 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c5d3a-50d4-4c16-9d5e-43cc12b6f162 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:19.288063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:19.349068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb5ace-564b-47a5-8c22-855ca8f768f9 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:19.393167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730834ec-5f51-4052-8547-265a297f2fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:19.393867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730834ec-5f51-4052-8547-265a297f2fd3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:19.394142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730834ec-5f51-4052-8547-265a297f2fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:19.394503 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730834ec-5f51-4052-8547-265a297f2fd3 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:19.401037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730834ec-5f51-4052-8547-265a297f2fd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:19.401411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730834ec-5f51-4052-8547-265a297f2fd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:19.403699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730834ec-5f51-4052-8547-265a297f2fd3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:19.418178 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c5d3a-50d4-4c16-9d5e-43cc12b6f162 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:19.418927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 119/228] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:36:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:19.425250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730834ec-5f51-4052-8547-265a297f2fd3 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:19.461041 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6decd0-01fe-4808-862a-57cdffafb661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:19.461943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6decd0-01fe-4808-862a-57cdffafb661 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:19.462228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6decd0-01fe-4808-862a-57cdffafb661 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:19.462504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6decd0-01fe-4808-862a-57cdffafb661 None admin] PATH_INFO: `/v3/users/db884278393f4de19359c6c1e560f1bd` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:19.464134 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6decd0-01fe-4808-862a-57cdffafb661 None admin] RBAC: Authorizing `identity:get_user(user_id=db884278393f4de19359c6c1e560f1bd)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:19.465763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6decd0-01fe-4808-862a-57cdffafb661 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:19.467972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 110/229] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:19 2022] GET /identity/v3/users/db884278393f4de19359c6c1e560f1bd => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:19.474145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6decd0-01fe-4808-862a-57cdffafb661 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:19.554030 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:19.613596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6decd0-01fe-4808-862a-57cdffafb661 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:19.658168 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ad80d-49af-4a5e-99a5-99e45fb843bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:19.659402 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ad80d-49af-4a5e-99a5-99e45fb843bb None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:19.659752 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ad80d-49af-4a5e-99a5-99e45fb843bb None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:19.660094 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ad80d-49af-4a5e-99a5-99e45fb843bb None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:19.661358 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ad80d-49af-4a5e-99a5-99e45fb843bb None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:19.663022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ad80d-49af-4a5e-99a5-99e45fb843bb None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:19.669825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 111/230] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:19 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:19.671495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ad80d-49af-4a5e-99a5-99e45fb843bb None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:19.708910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730834ec-5f51-4052-8547-265a297f2fd3 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:19.767555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b77288-602c-4204-97ca-c0f6d556003b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:19.768198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b77288-602c-4204-97ca-c0f6d556003b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:19.768471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b77288-602c-4204-97ca-c0f6d556003b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:19.768739 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b77288-602c-4204-97ca-c0f6d556003b None admin] PATH_INFO: `/v3/users/db884278393f4de19359c6c1e560f1bd` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:19.770371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b77288-602c-4204-97ca-c0f6d556003b None admin] RBAC: Authorizing `identity:get_user(user_id=db884278393f4de19359c6c1e560f1bd)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:19.773014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b77288-602c-4204-97ca-c0f6d556003b None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:19.775902 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 120/231] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:19 2022] GET /identity/v3/users/db884278393f4de19359c6c1e560f1bd => generated 283 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:19.782460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b77288-602c-4204-97ca-c0f6d556003b None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:19.785659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:19.858089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ad80d-49af-4a5e-99a5-99e45fb843bb None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:19.894972 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b77288-602c-4204-97ca-c0f6d556003b None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:19.915458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:19.927888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb30e2b-46e2-4f43-a9d7-4d86a0f3c8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:19.928594 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb30e2b-46e2-4f43-a9d7-4d86a0f3c8b9 None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:19.928939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb30e2b-46e2-4f43-a9d7-4d86a0f3c8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:19.929079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb30e2b-46e2-4f43-a9d7-4d86a0f3c8b9 None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7/users/db884278393f4de19359c6c1e560f1bd/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:19.934140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a461e-7a68-457b-8284-48386df834bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:19.935106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a461e-7a68-457b-8284-48386df834bd None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:19.935660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a461e-7a68-457b-8284-48386df834bd None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:19.936083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a461e-7a68-457b-8284-48386df834bd None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:19.936330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb30e2b-46e2-4f43-a9d7-4d86a0f3c8b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c33c6fa415942299549b8fdce48c1b7, user_id=db884278393f4de19359c6c1e560f1bd, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:19.937745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb30e2b-46e2-4f43-a9d7-4d86a0f3c8b9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:19.938152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a461e-7a68-457b-8284-48386df834bd None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:19.940663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a461e-7a68-457b-8284-48386df834bd None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:19.944008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 121/232] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:19 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:19.951204 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a461e-7a68-457b-8284-48386df834bd None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:19.960050 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 112/233] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:36:19 2022] PUT /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7/users/db884278393f4de19359c6c1e560f1bd/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:36:20.039710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:20.085470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a461e-7a68-457b-8284-48386df834bd None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:20.128647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae034c6-dca0-453a-96f5-c2bfdb22e46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:20.129337 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae034c6-dca0-453a-96f5-c2bfdb22e46e None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:20.129628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae034c6-dca0-453a-96f5-c2bfdb22e46e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:20.129890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae034c6-dca0-453a-96f5-c2bfdb22e46e None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7/users/db884278393f4de19359c6c1e560f1bd/roles/686764f0686e446f884315081e158bf3` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:20.135298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae034c6-dca0-453a-96f5-c2bfdb22e46e None admin] RBAC: Authorizing `identity:create_grant(project_id=5c33c6fa415942299549b8fdce48c1b7, user_id=db884278393f4de19359c6c1e560f1bd, role_id=686764f0686e446f884315081e158bf3)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:20.136995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae034c6-dca0-453a-96f5-c2bfdb22e46e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:20.167101 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 122/234] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:36:19 2022] PUT /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7/users/db884278393f4de19359c6c1e560f1bd/roles/686764f0686e446f884315081e158bf3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:36:21.453607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81189645-e510-468c-a5f0-6e4a9581130e None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:21.454132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81189645-e510-468c-a5f0-6e4a9581130e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:21.454515 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81189645-e510-468c-a5f0-6e4a9581130e None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:21.456731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 113/235] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:21.468385 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c15031-5faa-46ff-81e3-b2aabe56b5d9 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:21.468924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c15031-5faa-46ff-81e3-b2aabe56b5d9 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:21.468924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c15031-5faa-46ff-81e3-b2aabe56b5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:21.496760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-68c15031-5faa-46ff-81e3-b2aabe56b5d9 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:21.497978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-68c15031-5faa-46ff-81e3-b2aabe56b5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:21.504207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b57b62-c0f3-4bfd-89b6-bff744090351 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:21.505061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b57b62-c0f3-4bfd-89b6-bff744090351 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:21.505783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b57b62-c0f3-4bfd-89b6-bff744090351 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:21.508852 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 114/236] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:21.522843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b94b68-d4b2-44ca-ad29-e348e5d1ce6d None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:21.523194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b94b68-d4b2-44ca-ad29-e348e5d1ce6d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:21.523561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b94b68-d4b2-44ca-ad29-e348e5d1ce6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:21.556331 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-56b94b68-d4b2-44ca-ad29-e348e5d1ce6d None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:21.558261 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-56b94b68-d4b2-44ca-ad29-e348e5d1ce6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:21.648922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 123/237] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:21.677368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 115/238] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:21.692897 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c15031-5faa-46ff-81e3-b2aabe56b5d9 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:21.736655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b94b68-d4b2-44ca-ad29-e348e5d1ce6d None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:21.818010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:21.905577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c15031-5faa-46ff-81e3-b2aabe56b5d9 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:21.946258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc879ea5-1216-4b07-a9e2-66ad63355fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:21.946949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc879ea5-1216-4b07-a9e2-66ad63355fe2 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:21.947225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc879ea5-1216-4b07-a9e2-66ad63355fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:21.947588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc879ea5-1216-4b07-a9e2-66ad63355fe2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:21.949209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc879ea5-1216-4b07-a9e2-66ad63355fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:21.951889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc879ea5-1216-4b07-a9e2-66ad63355fe2 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:21.953745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 124/239] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:21.956031 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b94b68-d4b2-44ca-ad29-e348e5d1ce6d None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:21.963202 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc879ea5-1216-4b07-a9e2-66ad63355fe2 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:21.981891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:22.028757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fc9e6-5a11-488a-af8a-eb20393cbf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:22.029428 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fc9e6-5a11-488a-af8a-eb20393cbf55 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:22.029701 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fc9e6-5a11-488a-af8a-eb20393cbf55 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:22.029970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fc9e6-5a11-488a-af8a-eb20393cbf55 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:22.060465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fc9e6-5a11-488a-af8a-eb20393cbf55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-358fc9e6-5a11-488a-af8a-eb20393cbf55 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:22.073772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:22.074762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:22.075581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:22.075581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:22.075581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:22.075581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:22.075581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:22.075581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:22.075581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:22.075581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:22.075581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:22.075581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:22.075581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:22.075581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:22.084339 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 116/240] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:22.085332 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fc9e6-5a11-488a-af8a-eb20393cbf55 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:22.163147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc879ea5-1216-4b07-a9e2-66ad63355fe2 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:22.204199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:22.215274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af28f1-d30e-4f67-937a-e149cd094ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:22.215906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af28f1-d30e-4f67-937a-e149cd094ba3 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:22.216349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af28f1-d30e-4f67-937a-e149cd094ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:22.216625 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af28f1-d30e-4f67-937a-e149cd094ba3 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:22.217699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af28f1-d30e-4f67-937a-e149cd094ba3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:22.220036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af28f1-d30e-4f67-937a-e149cd094ba3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:22.230007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fc9e6-5a11-488a-af8a-eb20393cbf55 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:22.250723 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 125/241] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:36:21 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:36:22.252259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db0489-ad92-4306-be67-3eb1edee4ed0 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:22.252650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db0489-ad92-4306-be67-3eb1edee4ed0 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:22.252924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db0489-ad92-4306-be67-3eb1edee4ed0 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:22.264045 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 126/242] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:22.266766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c20170-8508-41fb-b26a-85555dd2f02a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:22.268954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c20170-8508-41fb-b26a-85555dd2f02a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:22.268954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c20170-8508-41fb-b26a-85555dd2f02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:22.316619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-49c20170-8508-41fb-b26a-85555dd2f02a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:22.319904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-49c20170-8508-41fb-b26a-85555dd2f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:22.324085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a8ca1-579f-4823-8d95-668b7ce75be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:22.324768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a8ca1-579f-4823-8d95-668b7ce75be6 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:22.325048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a8ca1-579f-4823-8d95-668b7ce75be6 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:22.325313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a8ca1-579f-4823-8d95-668b7ce75be6 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:22.326106 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a8ca1-579f-4823-8d95-668b7ce75be6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:22.326461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a8ca1-579f-4823-8d95-668b7ce75be6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:22.329703 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a8ca1-579f-4823-8d95-668b7ce75be6 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:22.348668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 117/243] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:22.356080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a8ca1-579f-4823-8d95-668b7ce75be6 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:22.427997 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 127/244] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:22.467188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:22.479655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c20170-8508-41fb-b26a-85555dd2f02a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:22.598616 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a8ca1-579f-4823-8d95-668b7ce75be6 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:22.640737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c20170-8508-41fb-b26a-85555dd2f02a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:22.650842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b2e33-e258-4444-b0b5-33ead7737938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:22.651493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b2e33-e258-4444-b0b5-33ead7737938 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:22.651774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b2e33-e258-4444-b0b5-33ead7737938 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:22.652045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b2e33-e258-4444-b0b5-33ead7737938 None admin] PATH_INFO: `/v3/users/3ef396b1e06e462882af64fa4874d910` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:22.658285 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:22.668537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b2e33-e258-4444-b0b5-33ead7737938 None admin] RBAC: Authorizing `identity:get_user(user_id=3ef396b1e06e462882af64fa4874d910)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:22.670042 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b2e33-e258-4444-b0b5-33ead7737938 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:22.672152 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 118/245] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:22 2022] GET /identity/v3/users/3ef396b1e06e462882af64fa4874d910 => generated 297 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:22.675241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50197b-5a39-45b1-8a6e-5c7f22ff1d77 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:22.676542 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50197b-5a39-45b1-8a6e-5c7f22ff1d77 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:22.677683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50197b-5a39-45b1-8a6e-5c7f22ff1d77 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:22.679296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 119/246] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:22.681077 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50197b-5a39-45b1-8a6e-5c7f22ff1d77 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:22.687358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729aa3a1-4b89-4dc0-9a15-22251d08d3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:22.688270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729aa3a1-4b89-4dc0-9a15-22251d08d3ac None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:22.688661 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729aa3a1-4b89-4dc0-9a15-22251d08d3ac None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:22.688930 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729aa3a1-4b89-4dc0-9a15-22251d08d3ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:22.706541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729aa3a1-4b89-4dc0-9a15-22251d08d3ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-729aa3a1-4b89-4dc0-9a15-22251d08d3ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:22.712571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:22.714209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 128/247] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:36:22.715525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9ca61-4487-4611-bf1f-cee861188001 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:22.716252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9ca61-4487-4611-bf1f-cee861188001 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:22.716252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9ca61-4487-4611-bf1f-cee861188001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:22.745270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-15b9ca61-4487-4611-bf1f-cee861188001 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:22.745270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-15b9ca61-4487-4611-bf1f-cee861188001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:22.824981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:22.835324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50197b-5a39-45b1-8a6e-5c7f22ff1d77 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:22.873506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 129/248] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:22.877413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9ca61-4487-4611-bf1f-cee861188001 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:22.927440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef568c-8fc2-44e7-875f-945c1e155d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:22.928066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef568c-8fc2-44e7-875f-945c1e155d5d None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:22.928343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef568c-8fc2-44e7-875f-945c1e155d5d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:22.928609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef568c-8fc2-44e7-875f-945c1e155d5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:23.066502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef568c-8fc2-44e7-875f-945c1e155d5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:23.068377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef568c-8fc2-44e7-875f-945c1e155d5d None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:23.088336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9ca61-4487-4611-bf1f-cee861188001 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:23.091075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:23.170013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31d4b3-ecc7-4eee-8d96-f66f9d795283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:23.170529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31d4b3-ecc7-4eee-8d96-f66f9d795283 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:23.170838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31d4b3-ecc7-4eee-8d96-f66f9d795283 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:23.171108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31d4b3-ecc7-4eee-8d96-f66f9d795283 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:23.171836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f31d4b3-ecc7-4eee-8d96-f66f9d795283 None admin] RBAC: Adding query filter params (name=member) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:23.172234 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f31d4b3-ecc7-4eee-8d96-f66f9d795283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:23.174442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f31d4b3-ecc7-4eee-8d96-f66f9d795283 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:23.191230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 130/249] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:23.191860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31d4b3-ecc7-4eee-8d96-f66f9d795283 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:23.221518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:23.233633 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 120/250] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:23.236249 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b3b15-ae69-46b8-a1e9-9fd3004b03bc None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:23.236542 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b3b15-ae69-46b8-a1e9-9fd3004b03bc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:23.236822 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b3b15-ae69-46b8-a1e9-9fd3004b03bc None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:23.241002 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 121/251] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:23.241002 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b3b15-ae69-46b8-a1e9-9fd3004b03bc None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:23.330311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31d4b3-ecc7-4eee-8d96-f66f9d795283 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:23.377931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d2360-0847-4357-8d61-aa353f515c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:23.378946 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d2360-0847-4357-8d61-aa353f515c1b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:23.379368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d2360-0847-4357-8d61-aa353f515c1b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:23.379779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d2360-0847-4357-8d61-aa353f515c1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:23.388199 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b3b15-ae69-46b8-a1e9-9fd3004b03bc None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:23.394474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d2360-0847-4357-8d61-aa353f515c1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-b85d2360-0847-4357-8d61-aa353f515c1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:23.401762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:23.402914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:23.403707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:23.403707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:23.403707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:23.403707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:23.403707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:23.403707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:23.403707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:23.403707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:23.403707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:23.403707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:23.403707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:23.403707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:23.405970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 131/252] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:36:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:23.411929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d2360-0847-4357-8d61-aa353f515c1b None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:23.430828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f372ff-87f9-4479-bd72-936eba1d347b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:23.431018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f372ff-87f9-4479-bd72-936eba1d347b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:23.431351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f372ff-87f9-4479-bd72-936eba1d347b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:23.431691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f372ff-87f9-4479-bd72-936eba1d347b None admin] PATH_INFO: `/v3/users/8b0baa8bb10844e4bde9c20839ed5733` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:23.433446 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f372ff-87f9-4479-bd72-936eba1d347b None admin] RBAC: Authorizing `identity:get_user(user_id=8b0baa8bb10844e4bde9c20839ed5733)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:23.435949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f372ff-87f9-4479-bd72-936eba1d347b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:23.438152 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 122/253] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:23 2022] GET /identity/v3/users/8b0baa8bb10844e4bde9c20839ed5733 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:23.444880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7aa2f-e99b-4743-a78e-b5e552c9d748 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:23.445690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7aa2f-e99b-4743-a78e-b5e552c9d748 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:23.446129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7aa2f-e99b-4743-a78e-b5e552c9d748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:23.451612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:23.473201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-01c7aa2f-e99b-4743-a78e-b5e552c9d748 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:23.474836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-01c7aa2f-e99b-4743-a78e-b5e552c9d748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:23.552086 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:23.560757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d2360-0847-4357-8d61-aa353f515c1b None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:23.566282 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 123/254] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:23.567433 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7aa2f-e99b-4743-a78e-b5e552c9d748 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:23.619984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9853300-b6ec-4cd0-85cc-aba2637928ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:23.621135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9853300-b6ec-4cd0-85cc-aba2637928ce None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:23.621660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9853300-b6ec-4cd0-85cc-aba2637928ce None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:23.622244 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9853300-b6ec-4cd0-85cc-aba2637928ce None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:23.625587 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9853300-b6ec-4cd0-85cc-aba2637928ce None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:23.627555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9853300-b6ec-4cd0-85cc-aba2637928ce None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:23.630249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 132/255] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:23 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:23.630906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9853300-b6ec-4cd0-85cc-aba2637928ce None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:23.707478 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7aa2f-e99b-4743-a78e-b5e552c9d748 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:23.765286 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa7081-1800-4f70-b9d0-35569b532672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:23.765921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa7081-1800-4f70-b9d0-35569b532672 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:23.766198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa7081-1800-4f70-b9d0-35569b532672 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:23.766467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa7081-1800-4f70-b9d0-35569b532672 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:23.767349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:23.768219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa7081-1800-4f70-b9d0-35569b532672 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:23.768344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa7081-1800-4f70-b9d0-35569b532672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:23.770467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa7081-1800-4f70-b9d0-35569b532672 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:23.777800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9853300-b6ec-4cd0-85cc-aba2637928ce None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:23.783799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 124/256] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:23.788516 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa7081-1800-4f70-b9d0-35569b532672 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:23.827665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae6e84-7fb3-44f3-828e-5c0f811b7881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:23.828722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae6e84-7fb3-44f3-828e-5c0f811b7881 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:23.829204 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae6e84-7fb3-44f3-828e-5c0f811b7881 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:23.829748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae6e84-7fb3-44f3-828e-5c0f811b7881 None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:23.831405 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae6e84-7fb3-44f3-828e-5c0f811b7881 None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:23.833126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae6e84-7fb3-44f3-828e-5c0f811b7881 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:23.841078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 133/257] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:23 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:23.842711 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae6e84-7fb3-44f3-828e-5c0f811b7881 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:23.885386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:23.941436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa7081-1800-4f70-b9d0-35569b532672 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:23.988628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20767a-c4fb-40cf-b439-85307e18cda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:23.989613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20767a-c4fb-40cf-b439-85307e18cda0 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:23.990097 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20767a-c4fb-40cf-b439-85307e18cda0 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:23.990592 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20767a-c4fb-40cf-b439-85307e18cda0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:23.995613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae6e84-7fb3-44f3-828e-5c0f811b7881 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:24.006530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20767a-c4fb-40cf-b439-85307e18cda0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-fc20767a-c4fb-40cf-b439-85307e18cda0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:24.015560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:24.016596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:24.017426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:24.017426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:24.017426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:24.017426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:24.017426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:24.017426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:24.017426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:24.017426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:24.017426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:24.017426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:24.017426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:24.017426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:24.020506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 125/258] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:36:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:24.022351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20767a-c4fb-40cf-b439-85307e18cda0 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:24.041804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2adcec-6fe4-460d-9ebb-1653769d5e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:24.042430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2adcec-6fe4-460d-9ebb-1653769d5e51 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:24.042772 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2adcec-6fe4-460d-9ebb-1653769d5e51 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:24.043028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2adcec-6fe4-460d-9ebb-1653769d5e51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:24.043798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2adcec-6fe4-460d-9ebb-1653769d5e51 None admin] RBAC: Adding query filter params (role.id=78132e8886fd4da988a6fe763a388e16, scope.project.id=5c33c6fa415942299549b8fdce48c1b7, user.id=3ef396b1e06e462882af64fa4874d910) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:24.044099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2adcec-6fe4-460d-9ebb-1653769d5e51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:24.046036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2adcec-6fe4-460d-9ebb-1653769d5e51 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:24.053767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 134/259] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:23 2022] GET /identity/v3/role_assignments?user.id=3ef396b1e06e462882af64fa4874d910&scope.project.id=5c33c6fa415942299549b8fdce48c1b7&role.id=78132e8886fd4da988a6fe763a388e16 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:24.054349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2adcec-6fe4-460d-9ebb-1653769d5e51 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:24.126675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:24.252947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20767a-c4fb-40cf-b439-85307e18cda0 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:24.278888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:24.285192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2adcec-6fe4-460d-9ebb-1653769d5e51 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:24.302465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235994dc-4b6a-459c-b605-28d68ba28a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:24.303160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235994dc-4b6a-459c-b605-28d68ba28a2c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:24.303442 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235994dc-4b6a-459c-b605-28d68ba28a2c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:24.303717 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235994dc-4b6a-459c-b605-28d68ba28a2c None admin] PATH_INFO: `/v3/users/db884278393f4de19359c6c1e560f1bd` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:24.307950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235994dc-4b6a-459c-b605-28d68ba28a2c None admin] RBAC: Authorizing `identity:get_user(user_id=db884278393f4de19359c6c1e560f1bd)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:24.309471 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235994dc-4b6a-459c-b605-28d68ba28a2c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:24.311271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 126/260] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:24 2022] GET /identity/v3/users/db884278393f4de19359c6c1e560f1bd => generated 283 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:24.312139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235994dc-4b6a-459c-b605-28d68ba28a2c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:24.337714 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051de00-d27b-4f8b-a944-7f9f69394270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:24.338393 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051de00-d27b-4f8b-a944-7f9f69394270 None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:24.339174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051de00-d27b-4f8b-a944-7f9f69394270 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:24.339451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051de00-d27b-4f8b-a944-7f9f69394270 None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7/users/8b0baa8bb10844e4bde9c20839ed5733/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:24.341719 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051de00-d27b-4f8b-a944-7f9f69394270 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c33c6fa415942299549b8fdce48c1b7, user_id=8b0baa8bb10844e4bde9c20839ed5733, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:24.343398 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051de00-d27b-4f8b-a944-7f9f69394270 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:24.361873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 135/261] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:36:24 2022] PUT /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7/users/8b0baa8bb10844e4bde9c20839ed5733/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:36:24.362801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051de00-d27b-4f8b-a944-7f9f69394270 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:24.576833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:24.626832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051de00-d27b-4f8b-a944-7f9f69394270 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:24.710768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235994dc-4b6a-459c-b605-28d68ba28a2c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:24.715915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706714b0-e541-4621-ab41-fd4a7d51e524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:24.716572 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706714b0-e541-4621-ab41-fd4a7d51e524 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:24.716849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706714b0-e541-4621-ab41-fd4a7d51e524 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:24.717114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706714b0-e541-4621-ab41-fd4a7d51e524 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:24.838745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a66eaa-c7ac-444c-a64b-46dcfa924202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:24.838745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a66eaa-c7ac-444c-a64b-46dcfa924202 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:24.839229 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a66eaa-c7ac-444c-a64b-46dcfa924202 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:24.842039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a66eaa-c7ac-444c-a64b-46dcfa924202 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:24.842039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a66eaa-c7ac-444c-a64b-46dcfa924202 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:24.842039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a66eaa-c7ac-444c-a64b-46dcfa924202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:24.842039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a66eaa-c7ac-444c-a64b-46dcfa924202 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:24.866657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706714b0-e541-4621-ab41-fd4a7d51e524 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:24.870669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706714b0-e541-4621-ab41-fd4a7d51e524 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:24.879814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 127/262] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:36:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 568 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:24.898659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a66eaa-c7ac-444c-a64b-46dcfa924202 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:25.045071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:25.188897 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:25.246451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 136/263] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 884 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:25.250519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a66eaa-c7ac-444c-a64b-46dcfa924202 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:25.253821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9c75c-83a5-4ff2-8a88-09b5b7971395 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:25.254104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9c75c-83a5-4ff2-8a88-09b5b7971395 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:25.254425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9c75c-83a5-4ff2-8a88-09b5b7971395 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:25.256048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 137/264] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:25.261008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7abeb-0fc7-4443-91ff-9a497f166447 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:25.261008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7abeb-0fc7-4443-91ff-9a497f166447 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:25.261008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7abeb-0fc7-4443-91ff-9a497f166447 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:25.266354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 138/265] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:25.266354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7abeb-0fc7-4443-91ff-9a497f166447 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:25.310365 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb0f8b-fd45-419e-aa77-6f1e7b61ff0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:25.311333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb0f8b-fd45-419e-aa77-6f1e7b61ff0a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:25.311937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb0f8b-fd45-419e-aa77-6f1e7b61ff0a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:25.312383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb0f8b-fd45-419e-aa77-6f1e7b61ff0a None admin] PATH_INFO: `/v3/users/db884278393f4de19359c6c1e560f1bd` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:25.314481 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb0f8b-fd45-419e-aa77-6f1e7b61ff0a None admin] RBAC: Authorizing `identity:get_user(user_id=db884278393f4de19359c6c1e560f1bd)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:25.316208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb0f8b-fd45-419e-aa77-6f1e7b61ff0a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:25.318294 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 128/266] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:24 2022] GET /identity/v3/users/db884278393f4de19359c6c1e560f1bd => generated 283 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:25.319975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64370c13-8659-4c30-94f1-8e5dc76ee841 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:25.321247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64370c13-8659-4c30-94f1-8e5dc76ee841 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:25.321910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64370c13-8659-4c30-94f1-8e5dc76ee841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:25.345533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-64370c13-8659-4c30-94f1-8e5dc76ee841 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:25.347154 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-64370c13-8659-4c30-94f1-8e5dc76ee841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:25.401853 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7abeb-0fc7-4443-91ff-9a497f166447 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:25.439019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 129/267] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:25.440333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87570b3-66e4-4ef5-be09-010ae30196f8 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:25.440622 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87570b3-66e4-4ef5-be09-010ae30196f8 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:25.440897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87570b3-66e4-4ef5-be09-010ae30196f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:25.445981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d33e5-9124-469b-9a71-13a6a08d372e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:25.446772 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d33e5-9124-469b-9a71-13a6a08d372e None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:25.447126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d33e5-9124-469b-9a71-13a6a08d372e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:25.447474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d33e5-9124-469b-9a71-13a6a08d372e None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:25.448796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d33e5-9124-469b-9a71-13a6a08d372e None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:25.450702 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d33e5-9124-469b-9a71-13a6a08d372e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:25.452613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 139/268] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:25 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:25.453890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d33e5-9124-469b-9a71-13a6a08d372e None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:25.479036 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-d87570b3-66e4-4ef5-be09-010ae30196f8 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:25.480348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d87570b3-66e4-4ef5-be09-010ae30196f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:25.531834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:25.556672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:25.589543 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d33e5-9124-469b-9a71-13a6a08d372e None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:25.616701 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 130/269] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:25.619253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87570b3-66e4-4ef5-be09-010ae30196f8 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:25.634929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8f790-4f49-4e52-92e6-f85fb8306f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:25.636152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8f790-4f49-4e52-92e6-f85fb8306f99 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:25.636699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8f790-4f49-4e52-92e6-f85fb8306f99 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:25.637149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8f790-4f49-4e52-92e6-f85fb8306f99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:25.747976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8f790-4f49-4e52-92e6-f85fb8306f99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:25.749245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87570b3-66e4-4ef5-be09-010ae30196f8 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:25.750304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8f790-4f49-4e52-92e6-f85fb8306f99 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:25.797469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cf20c-6c63-4bbb-97cb-fd120f7344da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:25.798273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cf20c-6c63-4bbb-97cb-fd120f7344da None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:25.798638 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cf20c-6c63-4bbb-97cb-fd120f7344da None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:25.798997 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cf20c-6c63-4bbb-97cb-fd120f7344da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:25.799932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cf20c-6c63-4bbb-97cb-fd120f7344da None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=5c33c6fa415942299549b8fdce48c1b7, user.id=db884278393f4de19359c6c1e560f1bd) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:25.800317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cf20c-6c63-4bbb-97cb-fd120f7344da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:25.802002 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cf20c-6c63-4bbb-97cb-fd120f7344da None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:25.810140 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 131/270] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:25 2022] GET /identity/v3/role_assignments?user.id=db884278393f4de19359c6c1e560f1bd&scope.project.id=5c33c6fa415942299549b8fdce48c1b7&role.id=93eb204498fc49d297bcd1841e00818d => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:25.811165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cf20c-6c63-4bbb-97cb-fd120f7344da None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:25.837824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:25.913240 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 140/271] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:25.915655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:25.916958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8f790-4f49-4e52-92e6-f85fb8306f99 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:26.050522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cf20c-6c63-4bbb-97cb-fd120f7344da None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:26.106529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9326a6-0704-401e-a367-24e8723e8fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:26.107337 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9326a6-0704-401e-a367-24e8723e8fbb None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:26.107684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9326a6-0704-401e-a367-24e8723e8fbb None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:26.108024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9326a6-0704-401e-a367-24e8723e8fbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:26.121914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9326a6-0704-401e-a367-24e8723e8fbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:26.126318 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8f790-4f49-4e52-92e6-f85fb8306f99 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-fc9326a6-0704-401e-a367-24e8723e8fbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:26.127580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:26.128465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:26.129513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:26.129513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:26.129513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:26.129513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:26.129513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:26.129513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:26.129513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:26.129513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:26.129513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:26.129513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:26.129513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:26.129513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:26.129513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 132/272] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:26.130187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9326a6-0704-401e-a367-24e8723e8fbb None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:26.180346 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1265a6-ac9f-4555-9552-287f70da78c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:26.182310 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1265a6-ac9f-4555-9552-287f70da78c8 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:26.182831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1265a6-ac9f-4555-9552-287f70da78c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:26.183308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1265a6-ac9f-4555-9552-287f70da78c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:26.185418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1265a6-ac9f-4555-9552-287f70da78c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:26.187292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1265a6-ac9f-4555-9552-287f70da78c8 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:26.190867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 141/273] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:26.191968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1265a6-ac9f-4555-9552-287f70da78c8 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:26.239781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:26.304038 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:26.308791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9326a6-0704-401e-a367-24e8723e8fbb None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:26.347983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d013bf-48a7-49f4-9c60-a1d584be3198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:26.348631 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d013bf-48a7-49f4-9c60-a1d584be3198 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:26.348909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d013bf-48a7-49f4-9c60-a1d584be3198 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:26.349183 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d013bf-48a7-49f4-9c60-a1d584be3198 None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:26.350379 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d013bf-48a7-49f4-9c60-a1d584be3198 None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:26.353497 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d013bf-48a7-49f4-9c60-a1d584be3198 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:26.360354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1265a6-ac9f-4555-9552-287f70da78c8 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:26.360907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 133/274] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:26 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:26.361874 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d013bf-48a7-49f4-9c60-a1d584be3198 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:26.411409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5297e0-98bc-4737-8829-f29b338d512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:26.413266 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5297e0-98bc-4737-8829-f29b338d512f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:26.415041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5297e0-98bc-4737-8829-f29b338d512f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:26.415328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5297e0-98bc-4737-8829-f29b338d512f None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:26.416100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5297e0-98bc-4737-8829-f29b338d512f None admin] RBAC: Adding query filter params (name=member) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:26.416411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5297e0-98bc-4737-8829-f29b338d512f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:26.418237 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5297e0-98bc-4737-8829-f29b338d512f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:26.429310 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 142/275] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:26.434958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5297e0-98bc-4737-8829-f29b338d512f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:26.516949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d013bf-48a7-49f4-9c60-a1d584be3198 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:26.553551 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:26.567826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f75f30-bc21-46b9-9543-1f914297ef23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:26.568580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f75f30-bc21-46b9-9543-1f914297ef23 None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:26.568959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f75f30-bc21-46b9-9543-1f914297ef23 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:26.569330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f75f30-bc21-46b9-9543-1f914297ef23 None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:26.570659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f75f30-bc21-46b9-9543-1f914297ef23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:26.572342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f75f30-bc21-46b9-9543-1f914297ef23 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:26.576076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5297e0-98bc-4737-8829-f29b338d512f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:26.602035 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 134/276] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:36:26 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:36:26.603600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f75f30-bc21-46b9-9543-1f914297ef23 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:26.621623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8ebbb-ec39-40d7-bcc4-bdf1930e72d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:26.622267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8ebbb-ec39-40d7-bcc4-bdf1930e72d3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:26.622550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8ebbb-ec39-40d7-bcc4-bdf1930e72d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:26.622934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8ebbb-ec39-40d7-bcc4-bdf1930e72d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:26.624037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8ebbb-ec39-40d7-bcc4-bdf1930e72d3 None admin] RBAC: Adding query filter params (role.id=686764f0686e446f884315081e158bf3, scope.project.id=5c33c6fa415942299549b8fdce48c1b7, user.id=db884278393f4de19359c6c1e560f1bd) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:26.624581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8ebbb-ec39-40d7-bcc4-bdf1930e72d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:26.626234 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8ebbb-ec39-40d7-bcc4-bdf1930e72d3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:26.633636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 143/277] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:26 2022] GET /identity/v3/role_assignments?user.id=db884278393f4de19359c6c1e560f1bd&scope.project.id=5c33c6fa415942299549b8fdce48c1b7&role.id=686764f0686e446f884315081e158bf3 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:26.649917 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:26.791775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f75f30-bc21-46b9-9543-1f914297ef23 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:26.833621 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ef9d8-1d55-4889-a89f-2b12ffd90d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:26.834663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ef9d8-1d55-4889-a89f-2b12ffd90d2c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:26.835167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ef9d8-1d55-4889-a89f-2b12ffd90d2c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:26.835670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ef9d8-1d55-4889-a89f-2b12ffd90d2c None admin] PATH_INFO: `/v3/users/5449a9d30dff450d9c2960d4efc100e4` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:26.844289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32486c-b51f-41c7-84d3-2fd4b7d13549 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:26.844607 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32486c-b51f-41c7-84d3-2fd4b7d13549 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:26.844925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32486c-b51f-41c7-84d3-2fd4b7d13549 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:26.846448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 144/278] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:26.852893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0057da-ced5-4b38-8081-2189dc5d7661 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:26.853511 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0057da-ced5-4b38-8081-2189dc5d7661 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:26.853973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0057da-ced5-4b38-8081-2189dc5d7661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:26.854074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ef9d8-1d55-4889-a89f-2b12ffd90d2c None admin] RBAC: Authorizing `identity:get_user(user_id=5449a9d30dff450d9c2960d4efc100e4)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:26.856145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ef9d8-1d55-4889-a89f-2b12ffd90d2c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:26.858262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 135/279] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:26 2022] GET /identity/v3/users/5449a9d30dff450d9c2960d4efc100e4 => generated 293 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:26.866981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ef9d8-1d55-4889-a89f-2b12ffd90d2c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:26.876737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-4c0057da-ced5-4b38-8081-2189dc5d7661 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:26.878299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4c0057da-ced5-4b38-8081-2189dc5d7661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:26.977707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 145/280] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:26.981791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:27.005957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ef9d8-1d55-4889-a89f-2b12ffd90d2c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:27.007418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0057da-ced5-4b38-8081-2189dc5d7661 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:27.056677 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fd033-f1d9-4ebe-b62f-aa2024919024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:27.057308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fd033-f1d9-4ebe-b62f-aa2024919024 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:27.057587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fd033-f1d9-4ebe-b62f-aa2024919024 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:27.057921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fd033-f1d9-4ebe-b62f-aa2024919024 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:27.058759 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fd033-f1d9-4ebe-b62f-aa2024919024 None admin] RBAC: Adding query filter params (role.id=fe6c7625134a449c8395ab84ffd82371, user.id=5449a9d30dff450d9c2960d4efc100e4, scope.system=all) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:27.059089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fd033-f1d9-4ebe-b62f-aa2024919024 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:27.060544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fd033-f1d9-4ebe-b62f-aa2024919024 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:27.066714 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 136/281] 10.222.0.86 () {56 vars in 1308 bytes} [Fri Aug 5 19:36:26 2022] GET /identity/v3/role_assignments?user.id=5449a9d30dff450d9c2960d4efc100e4&scope.system=all&role.id=fe6c7625134a449c8395ab84ffd82371 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:27.136981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:27.168554 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0057da-ced5-4b38-8081-2189dc5d7661 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:27.206316 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d735e-c43a-4d69-af8b-acc3eb5fc3e2 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:27.206660 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d735e-c43a-4d69-af8b-acc3eb5fc3e2 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:27.206956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d735e-c43a-4d69-af8b-acc3eb5fc3e2 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:27.209662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 137/282] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:36:27.219038 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c3852-2d65-41ba-b0b2-3707bff5d145 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:27.219318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c3852-2d65-41ba-b0b2-3707bff5d145 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:27.219590 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c3852-2d65-41ba-b0b2-3707bff5d145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:27.220614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad792c-e78d-47a1-91d4-ec4c98e9bbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:27.221439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad792c-e78d-47a1-91d4-ec4c98e9bbdf None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:27.221865 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad792c-e78d-47a1-91d4-ec4c98e9bbdf None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:27.222278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad792c-e78d-47a1-91d4-ec4c98e9bbdf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:27.245179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad792c-e78d-47a1-91d4-ec4c98e9bbdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:27.249494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-bc5c3852-2d65-41ba-b0b2-3707bff5d145 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:27.251010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-bc5c3852-2d65-41ba-b0b2-3707bff5d145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-c8ad792c-e78d-47a1-91d4-ec4c98e9bbdf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:27.255228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:27.256387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:27.257297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:27.257297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:27.257297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:27.257297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:27.257297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:27.257297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:27.257297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:27.257297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:27.257297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:27.257297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:27.257297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:27.257297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:27.257297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 146/283] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:36:27.268582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad792c-e78d-47a1-91d4-ec4c98e9bbdf None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:27.334121 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 138/284] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:27.366014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c3852-2d65-41ba-b0b2-3707bff5d145 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:27.412596 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad792c-e78d-47a1-91d4-ec4c98e9bbdf None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:27.497662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d9b54-7504-48d0-8f08-8468cf8926ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:27.498292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9b54-7504-48d0-8f08-8468cf8926ec None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:27.498567 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9b54-7504-48d0-8f08-8468cf8926ec None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:27.498857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9b54-7504-48d0-8f08-8468cf8926ec None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:27.499580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d9b54-7504-48d0-8f08-8468cf8926ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:27.499878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d9b54-7504-48d0-8f08-8468cf8926ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:27.501284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d9b54-7504-48d0-8f08-8468cf8926ec None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:27.515037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:27.522781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 147/285] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:27.529689 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d9b54-7504-48d0-8f08-8468cf8926ec None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:27.545070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:27.595948 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c3852-2d65-41ba-b0b2-3707bff5d145 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:27.669765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9bff0-b60a-46d3-a09a-d369b5be22bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:27.670345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9bff0-b60a-46d3-a09a-d369b5be22bd None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:27.670642 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9bff0-b60a-46d3-a09a-d369b5be22bd None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:27.670913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9bff0-b60a-46d3-a09a-d369b5be22bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:27.690761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d9b54-7504-48d0-8f08-8468cf8926ec None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:27.703784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9bff0-b60a-46d3-a09a-d369b5be22bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-eef9bff0-b60a-46d3-a09a-d369b5be22bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:27.708831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:27.709756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:27.710560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:27.710560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:27.710560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:27.710560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:27.710560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:27.710560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:27.710560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:27.710560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:27.710560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:27.710560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:27.710560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:27.710560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:27.712799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 139/286] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:36:27.720918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9bff0-b60a-46d3-a09a-d369b5be22bd None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:27.748552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d59dd2-767f-483f-9255-791e0ec566dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:27.749207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d59dd2-767f-483f-9255-791e0ec566dd None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:27.750351 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d59dd2-767f-483f-9255-791e0ec566dd None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:27.750636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d59dd2-767f-483f-9255-791e0ec566dd None admin] PATH_INFO: `/v3/users/3ef396b1e06e462882af64fa4874d910` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:27.752007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d59dd2-767f-483f-9255-791e0ec566dd None admin] RBAC: Authorizing `identity:get_user(user_id=3ef396b1e06e462882af64fa4874d910)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:27.754877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d59dd2-767f-483f-9255-791e0ec566dd None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:27.762265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 148/287] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:27 2022] GET /identity/v3/users/3ef396b1e06e462882af64fa4874d910 => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:27.767226 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d59dd2-767f-483f-9255-791e0ec566dd None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:27.915032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:27.927488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9bff0-b60a-46d3-a09a-d369b5be22bd None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:27.978261 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adf1e6-fb79-4f56-a060-936052393071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:27.978939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adf1e6-fb79-4f56-a060-936052393071 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:27.979469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adf1e6-fb79-4f56-a060-936052393071 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:27.979884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adf1e6-fb79-4f56-a060-936052393071 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:27.980797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17adf1e6-fb79-4f56-a060-936052393071 None admin] RBAC: Adding query filter params (name=member) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:27.981102 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17adf1e6-fb79-4f56-a060-936052393071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:27.982511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17adf1e6-fb79-4f56-a060-936052393071 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:27.984805 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:27.994296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 140/288] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:28.000035 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adf1e6-fb79-4f56-a060-936052393071 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:28.020284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d59dd2-767f-483f-9255-791e0ec566dd None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:28.136874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c33254-bfa0-4d36-bcfd-0e7b7b85c95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:28.137671 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c33254-bfa0-4d36-bcfd-0e7b7b85c95b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:28.138088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c33254-bfa0-4d36-bcfd-0e7b7b85c95b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:28.138371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c33254-bfa0-4d36-bcfd-0e7b7b85c95b None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:28.143377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c33254-bfa0-4d36-bcfd-0e7b7b85c95b None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:28.144841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c33254-bfa0-4d36-bcfd-0e7b7b85c95b None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:28.146962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 149/289] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:27 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:28.153750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adf1e6-fb79-4f56-a060-936052393071 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:28.154780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c33254-bfa0-4d36-bcfd-0e7b7b85c95b None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:28.206907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445f12a-015e-4f83-b591-84b8c40853cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:28.207441 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445f12a-015e-4f83-b591-84b8c40853cc None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:28.207713 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445f12a-015e-4f83-b591-84b8c40853cc None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:28.207980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445f12a-015e-4f83-b591-84b8c40853cc None admin] PATH_INFO: `/v3/users/8b0baa8bb10844e4bde9c20839ed5733` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:28.209780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445f12a-015e-4f83-b591-84b8c40853cc None admin] RBAC: Authorizing `identity:get_user(user_id=8b0baa8bb10844e4bde9c20839ed5733)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:28.211933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445f12a-015e-4f83-b591-84b8c40853cc None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:28.215142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 141/290] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:27 2022] GET /identity/v3/users/8b0baa8bb10844e4bde9c20839ed5733 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:28.220917 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445f12a-015e-4f83-b591-84b8c40853cc None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:28.375262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:28.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c33254-bfa0-4d36-bcfd-0e7b7b85c95b None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:28.509122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445f12a-015e-4f83-b591-84b8c40853cc None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:28.534394 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284acab-0d4d-44d2-9688-c98d5e24800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:28.535122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284acab-0d4d-44d2-9688-c98d5e24800c None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:28.535505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284acab-0d4d-44d2-9688-c98d5e24800c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:28.535783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284acab-0d4d-44d2-9688-c98d5e24800c None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7/users/3ef396b1e06e462882af64fa4874d910/roles/78132e8886fd4da988a6fe763a388e16` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:28.542760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284acab-0d4d-44d2-9688-c98d5e24800c None admin] RBAC: Authorizing `identity:create_grant(project_id=5c33c6fa415942299549b8fdce48c1b7, user_id=3ef396b1e06e462882af64fa4874d910, role_id=78132e8886fd4da988a6fe763a388e16)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:28.544520 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284acab-0d4d-44d2-9688-c98d5e24800c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:28.548300 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:28.582002 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 150/291] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:36:28 2022] PUT /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7/users/3ef396b1e06e462882af64fa4874d910/roles/78132e8886fd4da988a6fe763a388e16 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:36:28.592463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65584e5f-18b5-4bfe-84bc-436b8089ac17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:28.593552 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65584e5f-18b5-4bfe-84bc-436b8089ac17 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:28.594042 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65584e5f-18b5-4bfe-84bc-436b8089ac17 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:28.594582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65584e5f-18b5-4bfe-84bc-436b8089ac17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:28.727381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65584e5f-18b5-4bfe-84bc-436b8089ac17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:28.728934 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65584e5f-18b5-4bfe-84bc-436b8089ac17 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:28.810139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:28.879803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 142/292] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:28.887268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284acab-0d4d-44d2-9688-c98d5e24800c None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:29.077541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284acab-0d4d-44d2-9688-c98d5e24800c None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:29.144223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85113aad-2d21-4711-ac8c-7878fbcb57b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:29.145188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85113aad-2d21-4711-ac8c-7878fbcb57b7 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:29.145614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85113aad-2d21-4711-ac8c-7878fbcb57b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:29.146024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85113aad-2d21-4711-ac8c-7878fbcb57b7 None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:29.149587 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85113aad-2d21-4711-ac8c-7878fbcb57b7 None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:29.151954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85113aad-2d21-4711-ac8c-7878fbcb57b7 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:29.155505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 151/293] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:28 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:29.161888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65584e5f-18b5-4bfe-84bc-436b8089ac17 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:29.307456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65584e5f-18b5-4bfe-84bc-436b8089ac17 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:29.380705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8875ab-dfaf-4a1b-8e0b-3670a3ae47fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:29.381772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8875ab-dfaf-4a1b-8e0b-3670a3ae47fe None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:29.382265 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8875ab-dfaf-4a1b-8e0b-3670a3ae47fe None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:29.383303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8875ab-dfaf-4a1b-8e0b-3670a3ae47fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:29.384934 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8875ab-dfaf-4a1b-8e0b-3670a3ae47fe None admin] RBAC: Adding query filter params (role.id=fe6c7625134a449c8395ab84ffd82371, scope.project.id=5c33c6fa415942299549b8fdce48c1b7, user.id=8b0baa8bb10844e4bde9c20839ed5733) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:29.385486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8875ab-dfaf-4a1b-8e0b-3670a3ae47fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:29.388267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8875ab-dfaf-4a1b-8e0b-3670a3ae47fe None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:29.400902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 143/294] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:29 2022] GET /identity/v3/role_assignments?user.id=8b0baa8bb10844e4bde9c20839ed5733&scope.project.id=5c33c6fa415942299549b8fdce48c1b7&role.id=fe6c7625134a449c8395ab84ffd82371 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:29.483924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a057ad4-a6d2-4e84-b88b-8ec6df882fe6 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:29.484453 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a057ad4-a6d2-4e84-b88b-8ec6df882fe6 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:29.484872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a057ad4-a6d2-4e84-b88b-8ec6df882fe6 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:29.486600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 152/295] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:29.493530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58005b-fa82-4e55-9420-3e7d4cdeb384 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:29.493887 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58005b-fa82-4e55-9420-3e7d4cdeb384 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:29.494168 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58005b-fa82-4e55-9420-3e7d4cdeb384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:29.528661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-3a58005b-fa82-4e55-9420-3e7d4cdeb384 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:29.530362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3a58005b-fa82-4e55-9420-3e7d4cdeb384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:29.567147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:29.634742 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 144/296] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:29.659946 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a057ad4-a6d2-4e84-b88b-8ec6df882fe6 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:29.746094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:29.794501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a057ad4-a6d2-4e84-b88b-8ec6df882fe6 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:29.818372 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255dc8e-d0a7-4bd3-b9e7-96695cbbf78e None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:29.818696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255dc8e-d0a7-4bd3-b9e7-96695cbbf78e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:29.818943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255dc8e-d0a7-4bd3-b9e7-96695cbbf78e None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:29.820861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 145/297] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:36:29.827540 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cae37-d3bf-4e87-952a-f9659bd6b3ff None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:29.828180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cae37-d3bf-4e87-952a-f9659bd6b3ff None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:29.828645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cae37-d3bf-4e87-952a-f9659bd6b3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:29.845203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207f93b-e9c8-4188-a70e-7913b3c6e7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:29.845203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207f93b-e9c8-4188-a70e-7913b3c6e7bd None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:29.845470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207f93b-e9c8-4188-a70e-7913b3c6e7bd None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:29.845570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207f93b-e9c8-4188-a70e-7913b3c6e7bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:29.846788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207f93b-e9c8-4188-a70e-7913b3c6e7bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:29.848245 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207f93b-e9c8-4188-a70e-7913b3c6e7bd None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:29.850183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 153/298] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:29.856686 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207f93b-e9c8-4188-a70e-7913b3c6e7bd None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:29.864175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-8b9cae37-d3bf-4e87-952a-f9659bd6b3ff None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:29.866302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-8b9cae37-d3bf-4e87-952a-f9659bd6b3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:29.979519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 146/299] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:29.980842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffe77a-d7c3-4ce7-9be5-6a3da5d75bc5 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:29.981126 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffe77a-d7c3-4ce7-9be5-6a3da5d75bc5 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:29.981401 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffe77a-d7c3-4ce7-9be5-6a3da5d75bc5 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:29.983000 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 147/300] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:29.993951 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c4197-68c3-426f-b56f-714838f61697 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:29.994262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c4197-68c3-426f-b56f-714838f61697 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:29.994523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c4197-68c3-426f-b56f-714838f61697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:30.011661 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207f93b-e9c8-4188-a70e-7913b3c6e7bd None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:30.016506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-693c4197-68c3-426f-b56f-714838f61697 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:30.017727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-693c4197-68c3-426f-b56f-714838f61697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:30.047636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:30.055461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34919c-88c8-4c5d-8028-66dbdbf14247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:30.056051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34919c-88c8-4c5d-8028-66dbdbf14247 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:30.056326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34919c-88c8-4c5d-8028-66dbdbf14247 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:30.056594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34919c-88c8-4c5d-8028-66dbdbf14247 None admin] PATH_INFO: `/v3/groups` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:30.057256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34919c-88c8-4c5d-8028-66dbdbf14247 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:30.058850 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34919c-88c8-4c5d-8028-66dbdbf14247 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:30.076976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 154/301] 10.222.0.86 () {60 vars in 1134 bytes} [Fri Aug 5 19:36:29 2022] POST /identity/v3/groups => generated 229 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:36:30.077808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34919c-88c8-4c5d-8028-66dbdbf14247 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:30.114748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 148/302] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:30.144840 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c4197-68c3-426f-b56f-714838f61697 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:30.171681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:30.253695 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34919c-88c8-4c5d-8028-66dbdbf14247 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:30.297026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c4197-68c3-426f-b56f-714838f61697 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:30.311717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc9448-397d-4c15-8446-707e94f73d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:30.312752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc9448-397d-4c15-8446-707e94f73d06 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:30.313514 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc9448-397d-4c15-8446-707e94f73d06 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:30.313788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc9448-397d-4c15-8446-707e94f73d06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:30.326097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc9448-397d-4c15-8446-707e94f73d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-66cc9448-397d-4c15-8446-707e94f73d06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:30.332468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:30.333371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:30.334150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:30.334150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:30.334150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:30.334150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:30.334150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:30.334150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:30.334150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:30.334150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:30.334150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:30.334150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:30.334150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:30.334150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:30.334150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 155/303] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:30.341068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc9448-397d-4c15-8446-707e94f73d06 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:30.342218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36c585-0b4b-43c4-aa7c-208d5327185e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:30.343005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36c585-0b4b-43c4-aa7c-208d5327185e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:30.343390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36c585-0b4b-43c4-aa7c-208d5327185e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:30.343703 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36c585-0b4b-43c4-aa7c-208d5327185e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:30.357202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36c585-0b4b-43c4-aa7c-208d5327185e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-8e36c585-0b4b-43c4-aa7c-208d5327185e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:30.362273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:30.363391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:30.364353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:30.364353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:30.364353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:30.364353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:30.364353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:30.364353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:30.364353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:30.364353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:30.364353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:30.364353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:30.364353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:30.364353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:30.364353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 149/304] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:36:30.370879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36c585-0b4b-43c4-aa7c-208d5327185e None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:30.431229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:30.490940 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc9448-397d-4c15-8446-707e94f73d06 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:30.500798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:30.532418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36c585-0b4b-43c4-aa7c-208d5327185e None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:30.537503 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc5f1c-bc95-436d-81e2-04ed284268aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:30.538250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc5f1c-bc95-436d-81e2-04ed284268aa None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:30.538655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc5f1c-bc95-436d-81e2-04ed284268aa None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:30.539679 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc5f1c-bc95-436d-81e2-04ed284268aa None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:30.540524 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc5f1c-bc95-436d-81e2-04ed284268aa None admin] RBAC: Adding query filter params (name=member) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:30.540926 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc5f1c-bc95-436d-81e2-04ed284268aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:30.543235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc5f1c-bc95-436d-81e2-04ed284268aa None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:30.554119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 156/305] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:30.563924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc5f1c-bc95-436d-81e2-04ed284268aa None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:30.575630 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bedd2d-c7e6-4b5a-b397-0870e47d7e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:30.576604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bedd2d-c7e6-4b5a-b397-0870e47d7e46 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:30.577082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bedd2d-c7e6-4b5a-b397-0870e47d7e46 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:30.577593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bedd2d-c7e6-4b5a-b397-0870e47d7e46 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:30.578534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bedd2d-c7e6-4b5a-b397-0870e47d7e46 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:30.579083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bedd2d-c7e6-4b5a-b397-0870e47d7e46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:30.580691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bedd2d-c7e6-4b5a-b397-0870e47d7e46 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:30.596373 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 150/306] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:30.597660 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bedd2d-c7e6-4b5a-b397-0870e47d7e46 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:30.739766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc5f1c-bc95-436d-81e2-04ed284268aa None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:30.756552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:30.768245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bedd2d-c7e6-4b5a-b397-0870e47d7e46 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:30.782762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae1675-a7a4-4f4f-8832-037ddf802905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:30.783647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae1675-a7a4-4f4f-8832-037ddf802905 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:30.784019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae1675-a7a4-4f4f-8832-037ddf802905 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:30.784497 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae1675-a7a4-4f4f-8832-037ddf802905 None admin] PATH_INFO: `/v3/users/5449a9d30dff450d9c2960d4efc100e4` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:30.785726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae1675-a7a4-4f4f-8832-037ddf802905 None admin] RBAC: Authorizing `identity:get_user(user_id=5449a9d30dff450d9c2960d4efc100e4)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:30.787374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae1675-a7a4-4f4f-8832-037ddf802905 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:30.790833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 157/307] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:30 2022] GET /identity/v3/users/5449a9d30dff450d9c2960d4efc100e4 => generated 293 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:30.799253 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae1675-a7a4-4f4f-8832-037ddf802905 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:30.838190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b8126-78d5-46a1-99da-d49b209283dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:30.838954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b8126-78d5-46a1-99da-d49b209283dd None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:30.839239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b8126-78d5-46a1-99da-d49b209283dd None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:30.839515 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b8126-78d5-46a1-99da-d49b209283dd None admin] PATH_INFO: `/v3/users/8de27355f5454e90a0472b2c5b63bd1e` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:30.856318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:30.861372 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b8126-78d5-46a1-99da-d49b209283dd None admin] RBAC: Authorizing `identity:get_user(user_id=8de27355f5454e90a0472b2c5b63bd1e)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:30.863406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b8126-78d5-46a1-99da-d49b209283dd None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:30.866677 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 151/308] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:30 2022] GET /identity/v3/users/8de27355f5454e90a0472b2c5b63bd1e => generated 293 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:30.876207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b8126-78d5-46a1-99da-d49b209283dd None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:30.928443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae1675-a7a4-4f4f-8832-037ddf802905 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:30.974391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54066f3-925d-4ff2-b0dc-b8a754aea069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:30.975451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54066f3-925d-4ff2-b0dc-b8a754aea069 None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:30.975942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54066f3-925d-4ff2-b0dc-b8a754aea069 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:30.976488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54066f3-925d-4ff2-b0dc-b8a754aea069 None admin] PATH_INFO: `/v3/system/users/5449a9d30dff450d9c2960d4efc100e4/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:30.978981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54066f3-925d-4ff2-b0dc-b8a754aea069 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5449a9d30dff450d9c2960d4efc100e4, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:30.980885 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54066f3-925d-4ff2-b0dc-b8a754aea069 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:30.996289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 158/309] 10.222.0.86 () {58 vars in 1334 bytes} [Fri Aug 5 19:36:30 2022] PUT /identity/v3/system/users/5449a9d30dff450d9c2960d4efc100e4/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:36:31.026601 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b8126-78d5-46a1-99da-d49b209283dd None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:31.076759 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd97a3-28ab-439f-8c5e-107df38730d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:31.077758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd97a3-28ab-439f-8c5e-107df38730d0 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:31.078233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd97a3-28ab-439f-8c5e-107df38730d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:31.078782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd97a3-28ab-439f-8c5e-107df38730d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:31.079790 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd97a3-28ab-439f-8c5e-107df38730d0 None admin] RBAC: Adding query filter params (role.id=78132e8886fd4da988a6fe763a388e16, user.id=8de27355f5454e90a0472b2c5b63bd1e, scope.system=all) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:31.080285 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd97a3-28ab-439f-8c5e-107df38730d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:31.082112 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd97a3-28ab-439f-8c5e-107df38730d0 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:31.090989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 152/310] 10.222.0.86 () {56 vars in 1308 bytes} [Fri Aug 5 19:36:30 2022] GET /identity/v3/role_assignments?user.id=8de27355f5454e90a0472b2c5b63bd1e&scope.system=all&role.id=78132e8886fd4da988a6fe763a388e16 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:31.392891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580678c-889c-441b-b310-446f08c2b1bf None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:31.393805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580678c-889c-441b-b310-446f08c2b1bf None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:31.394241 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580678c-889c-441b-b310-446f08c2b1bf None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:31.396121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 159/311] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:31.403732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6459d4-0f27-4983-9e8e-649c0ad2b161 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:31.404615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6459d4-0f27-4983-9e8e-649c0ad2b161 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:31.405029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6459d4-0f27-4983-9e8e-649c0ad2b161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:31.426197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-fd6459d4-0f27-4983-9e8e-649c0ad2b161 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:31.427765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-fd6459d4-0f27-4983-9e8e-649c0ad2b161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:31.508986 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:31.521981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 153/312] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:31.547144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580678c-889c-441b-b310-446f08c2b1bf None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:31.604485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:31.673001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580678c-889c-441b-b310-446f08c2b1bf None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:31.710042 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1374c9a-54b8-4eca-8e39-455cffdb6860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:31.711314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1374c9a-54b8-4eca-8e39-455cffdb6860 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:31.711631 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1374c9a-54b8-4eca-8e39-455cffdb6860 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:31.711853 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1374c9a-54b8-4eca-8e39-455cffdb6860 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:31.723407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1374c9a-54b8-4eca-8e39-455cffdb6860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-d1374c9a-54b8-4eca-8e39-455cffdb6860 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:31.730922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:31.731869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:31.732907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:31.732907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:31.732907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:31.732907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:31.732907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:31.732907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:31.732907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:31.732907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:31.732907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:31.732907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:31.732907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:31.732907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:31.732907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 160/313] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:31.738283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6459d4-0f27-4983-9e8e-649c0ad2b161 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:31.861694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6459d4-0f27-4983-9e8e-649c0ad2b161 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:31.908329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d363e62-46c1-4526-83e7-7d53ed9c0c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:31.909031 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d363e62-46c1-4526-83e7-7d53ed9c0c6d None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:31.909323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d363e62-46c1-4526-83e7-7d53ed9c0c6d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:31.909767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d363e62-46c1-4526-83e7-7d53ed9c0c6d None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:31.911038 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d363e62-46c1-4526-83e7-7d53ed9c0c6d None admin] RBAC: Adding query filter params (name=reader) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:31.911550 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d363e62-46c1-4526-83e7-7d53ed9c0c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:31.914092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d363e62-46c1-4526-83e7-7d53ed9c0c6d None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:31.925788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 154/314] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:31.933240 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1374c9a-54b8-4eca-8e39-455cffdb6860 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:32.048781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1374c9a-54b8-4eca-8e39-455cffdb6860 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:32.058162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:32.087990 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e912f7-b30e-4644-a432-afa982aa7bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:32.088632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e912f7-b30e-4644-a432-afa982aa7bbb None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:32.088904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e912f7-b30e-4644-a432-afa982aa7bbb None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:32.089168 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e912f7-b30e-4644-a432-afa982aa7bbb None admin] PATH_INFO: `/v3/users/3ef396b1e06e462882af64fa4874d910` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:32.090414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e912f7-b30e-4644-a432-afa982aa7bbb None admin] RBAC: Authorizing `identity:get_user(user_id=3ef396b1e06e462882af64fa4874d910)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:32.092003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e912f7-b30e-4644-a432-afa982aa7bbb None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:32.093971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 161/315] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:31 2022] GET /identity/v3/users/3ef396b1e06e462882af64fa4874d910 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:32.099703 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d363e62-46c1-4526-83e7-7d53ed9c0c6d None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:32.183189 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:32.238089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d363e62-46c1-4526-83e7-7d53ed9c0c6d None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:32.279945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b37a70-3986-4d15-8432-e81abf1d1189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:32.280958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b37a70-3986-4d15-8432-e81abf1d1189 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:32.281367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b37a70-3986-4d15-8432-e81abf1d1189 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:32.281810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b37a70-3986-4d15-8432-e81abf1d1189 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:32.360251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b37a70-3986-4d15-8432-e81abf1d1189 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:32.361744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b37a70-3986-4d15-8432-e81abf1d1189 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:32.428719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:32.448179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77955f3-5464-4d84-9e8e-d79b81c378e2 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:32.448672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77955f3-5464-4d84-9e8e-d79b81c378e2 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:32.448672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77955f3-5464-4d84-9e8e-d79b81c378e2 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:32.450076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 162/316] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:36:32.455646 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828b107-18f7-4179-8d15-0a1c1ba16b88 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:32.455930 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828b107-18f7-4179-8d15-0a1c1ba16b88 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:32.456200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828b107-18f7-4179-8d15-0a1c1ba16b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:32.479805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-2828b107-18f7-4179-8d15-0a1c1ba16b88 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:32.482111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2828b107-18f7-4179-8d15-0a1c1ba16b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:32.495347 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 155/317] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:36:32.502001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b37a70-3986-4d15-8432-e81abf1d1189 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:32.598677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 163/318] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:32.622429 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828b107-18f7-4179-8d15-0a1c1ba16b88 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:32.643456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b37a70-3986-4d15-8432-e81abf1d1189 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:32.704198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb0ce3-2c60-4cb1-8dce-aab75738b6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:32.705338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb0ce3-2c60-4cb1-8dce-aab75738b6b1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:32.706368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb0ce3-2c60-4cb1-8dce-aab75738b6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:32.706900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb0ce3-2c60-4cb1-8dce-aab75738b6b1 None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:32.708652 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb0ce3-2c60-4cb1-8dce-aab75738b6b1 None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:32.712147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb0ce3-2c60-4cb1-8dce-aab75738b6b1 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:32.716285 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 156/319] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:32 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:32.727835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb0ce3-2c60-4cb1-8dce-aab75738b6b1 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:32.762919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828b107-18f7-4179-8d15-0a1c1ba16b88 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:32.764764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:32.766419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:32.814306 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05dfbbc-935c-4612-b099-b470eaaced42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:32.815375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dfbbc-935c-4612-b099-b470eaaced42 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:32.816004 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dfbbc-935c-4612-b099-b470eaaced42 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:32.816426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dfbbc-935c-4612-b099-b470eaaced42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:32.817913 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05dfbbc-935c-4612-b099-b470eaaced42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:32.819700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05dfbbc-935c-4612-b099-b470eaaced42 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:32.821683 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 164/320] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:32.829266 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05dfbbc-935c-4612-b099-b470eaaced42 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:32.870549 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb0ce3-2c60-4cb1-8dce-aab75738b6b1 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:32.928179 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eddbab-1df6-4fb6-8416-ecd4790544b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:32.929020 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eddbab-1df6-4fb6-8416-ecd4790544b1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:32.929850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eddbab-1df6-4fb6-8416-ecd4790544b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:32.930257 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eddbab-1df6-4fb6-8416-ecd4790544b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:32.931905 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eddbab-1df6-4fb6-8416-ecd4790544b1 None admin] RBAC: Adding query filter params (role.id=78132e8886fd4da988a6fe763a388e16, scope.project.id=5c33c6fa415942299549b8fdce48c1b7, user.id=3ef396b1e06e462882af64fa4874d910) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:32.932622 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eddbab-1df6-4fb6-8416-ecd4790544b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:32.934647 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eddbab-1df6-4fb6-8416-ecd4790544b1 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:32.951104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 157/321] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:36:32 2022] GET /identity/v3/role_assignments?user.id=3ef396b1e06e462882af64fa4874d910&scope.project.id=5c33c6fa415942299549b8fdce48c1b7&role.id=78132e8886fd4da988a6fe763a388e16 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:32.999544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05dfbbc-935c-4612-b099-b470eaaced42 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:33.061148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfbcfcd-b6fe-47c4-acea-6d8396a2bf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:33.061758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbcfcd-b6fe-47c4-acea-6d8396a2bf4e None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:33.062040 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbcfcd-b6fe-47c4-acea-6d8396a2bf4e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:33.062314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbcfcd-b6fe-47c4-acea-6d8396a2bf4e None admin] PATH_INFO: `/v3/groups` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:33.063293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfbcfcd-b6fe-47c4-acea-6d8396a2bf4e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:33.065100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfbcfcd-b6fe-47c4-acea-6d8396a2bf4e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:33.085023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 165/322] 10.222.0.86 () {60 vars in 1134 bytes} [Fri Aug 5 19:36:32 2022] POST /identity/v3/groups => generated 226 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:36:33.242135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221c250-5e44-4c8e-bd13-9b092cf13106 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:33.243013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221c250-5e44-4c8e-bd13-9b092cf13106 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:33.243535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221c250-5e44-4c8e-bd13-9b092cf13106 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:33.245563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 158/323] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:36:33.252934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96900654-9b4e-4fb9-a184-c130b519d945 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:33.253226 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96900654-9b4e-4fb9-a184-c130b519d945 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:33.253406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96900654-9b4e-4fb9-a184-c130b519d945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:33.278938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-96900654-9b4e-4fb9-a184-c130b519d945 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:33.280317 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-96900654-9b4e-4fb9-a184-c130b519d945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:33.318057 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:33.386330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a79d54-788f-4e4b-8647-b21f1e413de0 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:33.386691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a79d54-788f-4e4b-8647-b21f1e413de0 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:33.386938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a79d54-788f-4e4b-8647-b21f1e413de0 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:33.388537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 159/324] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:36:33.393450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 166/325] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:33.394060 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a872-c718-4e40-8700-69b499fce62a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:33.394344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a872-c718-4e40-8700-69b499fce62a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:33.394613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a872-c718-4e40-8700-69b499fce62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:33.424965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-28d6a872-c718-4e40-8700-69b499fce62a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:33.426167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-28d6a872-c718-4e40-8700-69b499fce62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:33.435991 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96900654-9b4e-4fb9-a184-c130b519d945 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:33.531388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 160/326] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:33.557100 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6a872-c718-4e40-8700-69b499fce62a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:33.586305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:33.616092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96900654-9b4e-4fb9-a184-c130b519d945 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:33.671165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e57e8-58bc-474a-ace8-1b1380041ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:33.672286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e57e8-58bc-474a-ace8-1b1380041ece None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:33.672799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e57e8-58bc-474a-ace8-1b1380041ece None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:33.673323 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e57e8-58bc-474a-ace8-1b1380041ece None admin] PATH_INFO: `/v3/roles/member` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:33.693058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e57e8-58bc-474a-ace8-1b1380041ece None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-370e57e8-58bc-474a-ace8-1b1380041ece None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:33.698763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:33.699721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:33.700508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:33.700508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:33.700508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:33.700508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:33.700508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:33.700508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:33.700508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:33.700508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:33.700508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:33.700508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:33.700508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:33.700508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:33.700508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 167/327] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:33.710333 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e57e8-58bc-474a-ace8-1b1380041ece None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:33.727684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6a872-c718-4e40-8700-69b499fce62a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:33.753499 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:33.788882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f946a62-1116-40e4-bd5a-b37d65623aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:33.790405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f946a62-1116-40e4-bd5a-b37d65623aca None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:33.790405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f946a62-1116-40e4-bd5a-b37d65623aca None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:33.790832 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f946a62-1116-40e4-bd5a-b37d65623aca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:33.803615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f946a62-1116-40e4-bd5a-b37d65623aca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-9f946a62-1116-40e4-bd5a-b37d65623aca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:33.810113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:33.811333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:33.812600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:33.812600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:33.812600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:33.812600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:33.812600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:33.812600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:33.812600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:33.812600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:33.812600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:33.812600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:33.812600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:33.812600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:33.815495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 161/328] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:36:33.821885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f946a62-1116-40e4-bd5a-b37d65623aca None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:33.880911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e57e8-58bc-474a-ace8-1b1380041ece None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:33.932640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050acf7-7952-4069-9d4c-9900cc5aca32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:33.933296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050acf7-7952-4069-9d4c-9900cc5aca32 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:33.933523 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050acf7-7952-4069-9d4c-9900cc5aca32 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:33.933796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050acf7-7952-4069-9d4c-9900cc5aca32 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:33.934527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050acf7-7952-4069-9d4c-9900cc5aca32 None admin] RBAC: Adding query filter params (name=member) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:33.934872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050acf7-7952-4069-9d4c-9900cc5aca32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:33.936451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050acf7-7952-4069-9d4c-9900cc5aca32 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:33.953902 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 168/329] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:33.959202 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050acf7-7952-4069-9d4c-9900cc5aca32 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:33.971511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:34.003544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f946a62-1116-40e4-bd5a-b37d65623aca None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:34.045235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd7d70-1e36-4336-9b41-29f5af3301a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:34.045752 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd7d70-1e36-4336-9b41-29f5af3301a8 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:34.046062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd7d70-1e36-4336-9b41-29f5af3301a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:34.046306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd7d70-1e36-4336-9b41-29f5af3301a8 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:34.047051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd7d70-1e36-4336-9b41-29f5af3301a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:34.047346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd7d70-1e36-4336-9b41-29f5af3301a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:34.048743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd7d70-1e36-4336-9b41-29f5af3301a8 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:34.059984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 162/330] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:34.068601 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd7d70-1e36-4336-9b41-29f5af3301a8 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:34.109943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:34.129983 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050acf7-7952-4069-9d4c-9900cc5aca32 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:34.205787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b4d0f-2924-446e-bc20-eef88f05257e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:34.206872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b4d0f-2924-446e-bc20-eef88f05257e None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:34.208477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b4d0f-2924-446e-bc20-eef88f05257e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:34.208754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b4d0f-2924-446e-bc20-eef88f05257e None admin] PATH_INFO: `/v3/users/5449a9d30dff450d9c2960d4efc100e4` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:34.210857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b4d0f-2924-446e-bc20-eef88f05257e None admin] RBAC: Authorizing `identity:get_user(user_id=5449a9d30dff450d9c2960d4efc100e4)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:34.212429 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b4d0f-2924-446e-bc20-eef88f05257e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:34.219222 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 169/331] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:33 2022] GET /identity/v3/users/5449a9d30dff450d9c2960d4efc100e4 => generated 293 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:34.229909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b4d0f-2924-446e-bc20-eef88f05257e None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:34.311314 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd7d70-1e36-4336-9b41-29f5af3301a8 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:34.385457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43e6a4-c8de-4821-9c77-a4b95dc8a867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:34.386127 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43e6a4-c8de-4821-9c77-a4b95dc8a867 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:34.386398 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43e6a4-c8de-4821-9c77-a4b95dc8a867 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:34.386688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43e6a4-c8de-4821-9c77-a4b95dc8a867 None admin] PATH_INFO: `/v3/users/8de27355f5454e90a0472b2c5b63bd1e` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:34.387947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43e6a4-c8de-4821-9c77-a4b95dc8a867 None admin] RBAC: Authorizing `identity:get_user(user_id=8de27355f5454e90a0472b2c5b63bd1e)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:34.389408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43e6a4-c8de-4821-9c77-a4b95dc8a867 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:34.391467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 163/332] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:34 2022] GET /identity/v3/users/8de27355f5454e90a0472b2c5b63bd1e => generated 293 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:34.399293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43e6a4-c8de-4821-9c77-a4b95dc8a867 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:34.408686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:34.425046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b4d0f-2924-446e-bc20-eef88f05257e None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:34.485589 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583796f-6c70-4d35-b357-eb9565b0f5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:34.486232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583796f-6c70-4d35-b357-eb9565b0f5a1 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:34.486509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583796f-6c70-4d35-b357-eb9565b0f5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:34.486809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583796f-6c70-4d35-b357-eb9565b0f5a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:34.487615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583796f-6c70-4d35-b357-eb9565b0f5a1 None admin] RBAC: Adding query filter params (role.id=fe6c7625134a449c8395ab84ffd82371, user.id=5449a9d30dff450d9c2960d4efc100e4, scope.system=all) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:34.487917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583796f-6c70-4d35-b357-eb9565b0f5a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:34.489367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583796f-6c70-4d35-b357-eb9565b0f5a1 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:34.511854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 170/333] 10.222.0.86 () {56 vars in 1308 bytes} [Fri Aug 5 19:36:34 2022] GET /identity/v3/role_assignments?user.id=5449a9d30dff450d9c2960d4efc100e4&scope.system=all&role.id=fe6c7625134a449c8395ab84ffd82371 => generated 510 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:34.626491 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43e6a4-c8de-4821-9c77-a4b95dc8a867 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:34.682495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c8d75-166f-41fd-a4d1-50bd9fb44c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:34.683212 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c8d75-166f-41fd-a4d1-50bd9fb44c6b None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:34.683486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c8d75-166f-41fd-a4d1-50bd9fb44c6b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:34.683756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c8d75-166f-41fd-a4d1-50bd9fb44c6b None admin] PATH_INFO: `/v3/system/users/8de27355f5454e90a0472b2c5b63bd1e/roles/78132e8886fd4da988a6fe763a388e16` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:34.685291 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c8d75-166f-41fd-a4d1-50bd9fb44c6b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8de27355f5454e90a0472b2c5b63bd1e, role_id=78132e8886fd4da988a6fe763a388e16)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:34.686737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c8d75-166f-41fd-a4d1-50bd9fb44c6b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:34.702527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 164/334] 10.222.0.86 () {58 vars in 1334 bytes} [Fri Aug 5 19:36:34 2022] PUT /identity/v3/system/users/8de27355f5454e90a0472b2c5b63bd1e/roles/78132e8886fd4da988a6fe763a388e16 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:36:36.107471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861db05b-43f1-45dd-a393-12b23b3334c1 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:36.108039 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861db05b-43f1-45dd-a393-12b23b3334c1 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:36.108039 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861db05b-43f1-45dd-a393-12b23b3334c1 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:36.109720 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 171/335] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:36.116124 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4428cb-655a-477c-8e05-bcba919510b3 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:36.116654 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4428cb-655a-477c-8e05-bcba919510b3 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:36.116654 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4428cb-655a-477c-8e05-bcba919510b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:36.143886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-0e4428cb-655a-477c-8e05-bcba919510b3 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:36.145390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-0e4428cb-655a-477c-8e05-bcba919510b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:36.156647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b27209-1e5e-47f8-92ad-b56acb8a0ca1 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:36.156965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b27209-1e5e-47f8-92ad-b56acb8a0ca1 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:36.157192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b27209-1e5e-47f8-92ad-b56acb8a0ca1 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:36.158541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 172/336] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:36:36.164293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf4953-f698-4b2a-b19f-1ee52d904ed1 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:36.164915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf4953-f698-4b2a-b19f-1ee52d904ed1 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:36.165397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf4953-f698-4b2a-b19f-1ee52d904ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:36.196151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-29bf4953-f698-4b2a-b19f-1ee52d904ed1 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:36.197349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-29bf4953-f698-4b2a-b19f-1ee52d904ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:36.213339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:36.257196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:36.267843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 165/337] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:36.325816 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 173/338] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:36.330055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4428cb-655a-477c-8e05-bcba919510b3 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:36.368831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf4953-f698-4b2a-b19f-1ee52d904ed1 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:36.522666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4428cb-655a-477c-8e05-bcba919510b3 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:36.570656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf4953-f698-4b2a-b19f-1ee52d904ed1 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:36.596226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20510282-2acc-4f10-89d8-636f4e516b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:36.596877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20510282-2acc-4f10-89d8-636f4e516b64 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:36.597151 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20510282-2acc-4f10-89d8-636f4e516b64 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:36.597418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20510282-2acc-4f10-89d8-636f4e516b64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:36.631684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20510282-2acc-4f10-89d8-636f4e516b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:36.632485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4f071-45e4-4b88-8328-07628bfbcc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:36.633457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4f071-45e4-4b88-8328-07628bfbcc5f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:36.633887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4f071-45e4-4b88-8328-07628bfbcc5f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:36.634324 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4f071-45e4-4b88-8328-07628bfbcc5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-20510282-2acc-4f10-89d8-636f4e516b64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:36.638319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:36.639533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:36.640559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:36.640559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:36.640559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:36.640559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:36.640559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:36.640559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:36.640559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:36.640559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:36.640559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:36.640559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:36.640559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:36.640559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:36.647335 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:36.647509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 166/339] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:36:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:36:36.648193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20510282-2acc-4f10-89d8-636f4e516b64 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:36.651872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4f071-45e4-4b88-8328-07628bfbcc5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-e1a4f071-45e4-4b88-8328-07628bfbcc5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:36.657541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:36.658657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:36.659667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:36.659667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:36.659667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:36.659667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:36.659667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:36.659667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:36.659667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:36.659667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:36.659667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:36.659667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:36.659667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:36.659667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:36.665885 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 174/340] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:36.672088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4f071-45e4-4b88-8328-07628bfbcc5f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:36.850366 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:36.862406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4f071-45e4-4b88-8328-07628bfbcc5f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:36.907583 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ae5f9-ab73-476b-ba44-a6a8151f6bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:36.908444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ae5f9-ab73-476b-ba44-a6a8151f6bdd None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:36.908879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ae5f9-ab73-476b-ba44-a6a8151f6bdd None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:36.909311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ae5f9-ab73-476b-ba44-a6a8151f6bdd None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:36.910175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ae5f9-ab73-476b-ba44-a6a8151f6bdd None admin] RBAC: Adding query filter params (name=member) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:36.910653 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ae5f9-ab73-476b-ba44-a6a8151f6bdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:36.912872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ae5f9-ab73-476b-ba44-a6a8151f6bdd None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:36.923952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 175/341] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:36.926089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b90fb-b924-4407-b09e-25cfc4dbd115 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:36.926543 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b90fb-b924-4407-b09e-25cfc4dbd115 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:36.926987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b90fb-b924-4407-b09e-25cfc4dbd115 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:36.929117 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 176/342] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:36.930709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b90fb-b924-4407-b09e-25cfc4dbd115 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:36.974598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20510282-2acc-4f10-89d8-636f4e516b64 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:37.010019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:37.063495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b90fb-b924-4407-b09e-25cfc4dbd115 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:37.063694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09de45f-b48b-4ab5-bc4a-93152168ef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:37.064638 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09de45f-b48b-4ab5-bc4a-93152168ef76 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:37.065102 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09de45f-b48b-4ab5-bc4a-93152168ef76 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:37.065569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09de45f-b48b-4ab5-bc4a-93152168ef76 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:37.067721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09de45f-b48b-4ab5-bc4a-93152168ef76 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:37.067721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09de45f-b48b-4ab5-bc4a-93152168ef76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:37.070413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09de45f-b48b-4ab5-bc4a-93152168ef76 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:37.092491 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 167/343] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:36:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:37.094108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647343b-daea-4d71-97ce-8afc8b8c3ca9 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:37.094399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647343b-daea-4d71-97ce-8afc8b8c3ca9 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:37.094695 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647343b-daea-4d71-97ce-8afc8b8c3ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:37.107709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3cbe91-6c83-4d9d-afc3-2f1c2faeb50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:37.109993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cbe91-6c83-4d9d-afc3-2f1c2faeb50d None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:37.110475 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cbe91-6c83-4d9d-afc3-2f1c2faeb50d None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:37.110962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cbe91-6c83-4d9d-afc3-2f1c2faeb50d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:37.127787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-1647343b-daea-4d71-97ce-8afc8b8c3ca9 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:37.129010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-1647343b-daea-4d71-97ce-8afc8b8c3ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:37.198834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3cbe91-6c83-4d9d-afc3-2f1c2faeb50d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:37.200598 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3cbe91-6c83-4d9d-afc3-2f1c2faeb50d None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:37.272262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 168/344] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:37.273559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647343b-daea-4d71-97ce-8afc8b8c3ca9 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:37.281439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:37.333999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 177/345] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:37.341362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3cbe91-6c83-4d9d-afc3-2f1c2faeb50d None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:37.402791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:37.473102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3cbe91-6c83-4d9d-afc3-2f1c2faeb50d None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:37.509103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647343b-daea-4d71-97ce-8afc8b8c3ca9 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:37.519952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c8f80-60b6-4874-a0e3-fa6f8c4042e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:37.521407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c8f80-60b6-4874-a0e3-fa6f8c4042e5 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:37.521848 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c8f80-60b6-4874-a0e3-fa6f8c4042e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:37.522272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c8f80-60b6-4874-a0e3-fa6f8c4042e5 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:37.523987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c8f80-60b6-4874-a0e3-fa6f8c4042e5 None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:37.526504 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c8f80-60b6-4874-a0e3-fa6f8c4042e5 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:37.529664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 178/346] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:37 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:37.530708 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c8f80-60b6-4874-a0e3-fa6f8c4042e5 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:37.570373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:37.589666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42408969-8e2a-43e7-bd1c-6d20fac2711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:37.590313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408969-8e2a-43e7-bd1c-6d20fac2711d None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:37.590591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408969-8e2a-43e7-bd1c-6d20fac2711d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:37.590875 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408969-8e2a-43e7-bd1c-6d20fac2711d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:37.662538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c8f80-60b6-4874-a0e3-fa6f8c4042e5 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:37.704748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6c5e1-286e-4ffc-8bd3-822f69290755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:37.705728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6c5e1-286e-4ffc-8bd3-822f69290755 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:37.706263 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6c5e1-286e-4ffc-8bd3-822f69290755 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:37.706719 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6c5e1-286e-4ffc-8bd3-822f69290755 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:37.719834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6c5e1-286e-4ffc-8bd3-822f69290755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-2ed6c5e1-286e-4ffc-8bd3-822f69290755 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:37.727140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:37.728322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:37.729352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:37.729352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:37.729352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:37.729352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:37.729352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:37.729352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:37.729352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:37.729352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:37.729352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:37.729352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:37.729352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:37.729352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:37.731402 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 179/347] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:37.732574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6c5e1-286e-4ffc-8bd3-822f69290755 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:37.783887 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42408969-8e2a-43e7-bd1c-6d20fac2711d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:37.785420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42408969-8e2a-43e7-bd1c-6d20fac2711d None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:37.845580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:37.861269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6c5e1-286e-4ffc-8bd3-822f69290755 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:37.925483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:37.935225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff6a4e-5282-4b63-b045-c77ebd6ce2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:37.937588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff6a4e-5282-4b63-b045-c77ebd6ce2b6 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:37.938358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff6a4e-5282-4b63-b045-c77ebd6ce2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:37.938971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff6a4e-5282-4b63-b045-c77ebd6ce2b6 None admin] PATH_INFO: `/v3/groups/f5504a89d5b343d7bf5dac61e3410018` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:37.954059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff6a4e-5282-4b63-b045-c77ebd6ce2b6 None admin] RBAC: Authorizing `identity:get_group(group_id=f5504a89d5b343d7bf5dac61e3410018)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:37.956941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff6a4e-5282-4b63-b045-c77ebd6ce2b6 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:37.960217 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 180/348] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:37 2022] GET /identity/v3/groups/f5504a89d5b343d7bf5dac61e3410018 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:37.963375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff6a4e-5282-4b63-b045-c77ebd6ce2b6 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:38.029016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 169/349] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 756 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:38.030388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42408969-8e2a-43e7-bd1c-6d20fac2711d None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:38.135338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff6a4e-5282-4b63-b045-c77ebd6ce2b6 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:38.188185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42408969-8e2a-43e7-bd1c-6d20fac2711d None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:38.194199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f093bc-561f-461f-aaed-13f418100aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:38.196054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f093bc-561f-461f-aaed-13f418100aa5 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:38.196342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f093bc-561f-461f-aaed-13f418100aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:38.196944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f093bc-561f-461f-aaed-13f418100aa5 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:38.198737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f093bc-561f-461f-aaed-13f418100aa5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:38.200179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f093bc-561f-461f-aaed-13f418100aa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:38.200839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f093bc-561f-461f-aaed-13f418100aa5 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:38.211493 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 181/350] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:38.213857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564452b2-f01d-4db5-9f17-9eb9621fd1df None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:38.213857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564452b2-f01d-4db5-9f17-9eb9621fd1df None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:38.218792 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564452b2-f01d-4db5-9f17-9eb9621fd1df None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:38.220754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 182/351] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:38.221510 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564452b2-f01d-4db5-9f17-9eb9621fd1df None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:38.238655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ed45d-6a99-4196-8693-aab7fc85cfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:38.239296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ed45d-6a99-4196-8693-aab7fc85cfba None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:38.239572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ed45d-6a99-4196-8693-aab7fc85cfba None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:38.239846 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ed45d-6a99-4196-8693-aab7fc85cfba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:38.240618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ed45d-6a99-4196-8693-aab7fc85cfba None admin] RBAC: Adding query filter params (group.id=f5504a89d5b343d7bf5dac61e3410018, role.id=fe6c7625134a449c8395ab84ffd82371, scope.project.id=11d6ae35861149a9a30b9eadb5380b85) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:38.240924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ed45d-6a99-4196-8693-aab7fc85cfba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:38.244003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ed45d-6a99-4196-8693-aab7fc85cfba None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:38.250290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:38.250678 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 170/352] 10.222.0.86 () {56 vars in 1376 bytes} [Fri Aug 5 19:36:38 2022] GET /identity/v3/role_assignments?group.id=f5504a89d5b343d7bf5dac61e3410018&scope.project.id=11d6ae35861149a9a30b9eadb5380b85&role.id=fe6c7625134a449c8395ab84ffd82371 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:38.252025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7baecd-137b-486a-9220-f3b74901fcd8 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:38.252362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7baecd-137b-486a-9220-f3b74901fcd8 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:38.252634 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7baecd-137b-486a-9220-f3b74901fcd8 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:38.254265 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 171/353] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:38.255088 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7baecd-137b-486a-9220-f3b74901fcd8 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:38.289649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:38.394706 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564452b2-f01d-4db5-9f17-9eb9621fd1df None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:38.433908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7baecd-137b-486a-9220-f3b74901fcd8 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:38.448151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d4222-fde0-40b5-8384-6b2941834bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:38.449185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d4222-fde0-40b5-8384-6b2941834bf9 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:38.449670 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d4222-fde0-40b5-8384-6b2941834bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:38.450147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d4222-fde0-40b5-8384-6b2941834bf9 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:38.451731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d4222-fde0-40b5-8384-6b2941834bf9 None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:38.453645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d4222-fde0-40b5-8384-6b2941834bf9 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:38.455834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 183/354] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:38 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:38.457245 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6a34b-931d-489b-8178-7383765f5421 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:38.457741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6a34b-931d-489b-8178-7383765f5421 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:38.458268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6a34b-931d-489b-8178-7383765f5421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:38.493276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-46b6a34b-931d-489b-8178-7383765f5421 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:38.494312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349a72e-385a-4d65-aee0-45473e7b62ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:38.494605 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-46b6a34b-931d-489b-8178-7383765f5421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:38.495048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349a72e-385a-4d65-aee0-45473e7b62ee None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:38.495376 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349a72e-385a-4d65-aee0-45473e7b62ee None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:38.495814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349a72e-385a-4d65-aee0-45473e7b62ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:38.588091 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 184/355] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:38.588270 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349a72e-385a-4d65-aee0-45473e7b62ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:38.589583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349a72e-385a-4d65-aee0-45473e7b62ee None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:38.590146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c0455-bbf2-4d3b-99e8-cbd0d3ae15ed None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:38.591756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c0455-bbf2-4d3b-99e8-cbd0d3ae15ed None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:38.592277 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c0455-bbf2-4d3b-99e8-cbd0d3ae15ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:38.614870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c91c0455-bbf2-4d3b-99e8-cbd0d3ae15ed None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:38.616412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c91c0455-bbf2-4d3b-99e8-cbd0d3ae15ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:38.625119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:38.629234 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:38.732269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 185/356] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:38.735207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c0455-bbf2-4d3b-99e8-cbd0d3ae15ed None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:38.756884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 172/357] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:38.759455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9598a-fedc-4a81-b5f5-9ff369496416 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:38.759741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9598a-fedc-4a81-b5f5-9ff369496416 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:38.760017 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9598a-fedc-4a81-b5f5-9ff369496416 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:38.765909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 173/358] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:38.766560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9598a-fedc-4a81-b5f5-9ff369496416 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:38.876145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c0455-bbf2-4d3b-99e8-cbd0d3ae15ed None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:38.901232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9598a-fedc-4a81-b5f5-9ff369496416 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:38.924233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:38.924678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f84a1-77a1-49aa-bc06-d9bd12c565c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:38.925878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f84a1-77a1-49aa-bc06-d9bd12c565c3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:38.926333 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f84a1-77a1-49aa-bc06-d9bd12c565c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:38.926812 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f84a1-77a1-49aa-bc06-d9bd12c565c3 None admin] PATH_INFO: `/v3/groups/f5504a89d5b343d7bf5dac61e3410018` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:38.928246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f84a1-77a1-49aa-bc06-d9bd12c565c3 None admin] RBAC: Authorizing `identity:get_group(group_id=f5504a89d5b343d7bf5dac61e3410018)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:38.932172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f84a1-77a1-49aa-bc06-d9bd12c565c3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:38.933198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 186/359] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:38 2022] GET /identity/v3/groups/f5504a89d5b343d7bf5dac61e3410018 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:38.934557 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f84a1-77a1-49aa-bc06-d9bd12c565c3 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:38.954562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6dd4fd-b2ea-4aa3-b455-90098fde41ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:38.955673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dd4fd-b2ea-4aa3-b455-90098fde41ef None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:38.955673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dd4fd-b2ea-4aa3-b455-90098fde41ef None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:38.955882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dd4fd-b2ea-4aa3-b455-90098fde41ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:38.957675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:38.974096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6dd4fd-b2ea-4aa3-b455-90098fde41ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-fd6dd4fd-b2ea-4aa3-b455-90098fde41ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:38.979916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:38.981459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 174/360] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:36:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:38.982902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea73bb3-a1c7-4281-ae0d-f64f7783bd8a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:38.984144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea73bb3-a1c7-4281-ae0d-f64f7783bd8a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:38.984144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea73bb3-a1c7-4281-ae0d-f64f7783bd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:39.003862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-2ea73bb3-a1c7-4281-ae0d-f64f7783bd8a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:39.005138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2ea73bb3-a1c7-4281-ae0d-f64f7783bd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:39.103359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f84a1-77a1-49aa-bc06-d9bd12c565c3 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:39.107450 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 175/361] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:39.108973 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea73bb3-a1c7-4281-ae0d-f64f7783bd8a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:39.143336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5f4e8-b77d-4202-ab3d-3d86efb1fede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:39.143961 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5f4e8-b77d-4202-ab3d-3d86efb1fede None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:39.144175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5f4e8-b77d-4202-ab3d-3d86efb1fede None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:39.144714 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da5f4e8-b77d-4202-ab3d-3d86efb1fede None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:39.145757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5f4e8-b77d-4202-ab3d-3d86efb1fede None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:39.147278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da5f4e8-b77d-4202-ab3d-3d86efb1fede None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:39.149947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 187/362] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:38 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:39.150690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5f4e8-b77d-4202-ab3d-3d86efb1fede None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:39.260783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:39.265311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea73bb3-a1c7-4281-ae0d-f64f7783bd8a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:39.275245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:39.289355 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da5f4e8-b77d-4202-ab3d-3d86efb1fede None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:39.324342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086fd69-193e-44a2-8e60-650111de5909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:39.324997 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086fd69-193e-44a2-8e60-650111de5909 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:39.325270 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086fd69-193e-44a2-8e60-650111de5909 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:39.325532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086fd69-193e-44a2-8e60-650111de5909 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:39.338187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086fd69-193e-44a2-8e60-650111de5909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:39.340553 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a44012-7d68-41f7-9245-5ec2870cefc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:39.341177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44012-7d68-41f7-9245-5ec2870cefc4 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:39.341501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44012-7d68-41f7-9245-5ec2870cefc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:39.341718 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44012-7d68-41f7-9245-5ec2870cefc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:39.342519 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a44012-7d68-41f7-9245-5ec2870cefc4 None admin] RBAC: Adding query filter params (group.id=f5504a89d5b343d7bf5dac61e3410018, role.id=686764f0686e446f884315081e158bf3, scope.project.id=11d6ae35861149a9a30b9eadb5380b85) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:39.342857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a44012-7d68-41f7-9245-5ec2870cefc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-d086fd69-193e-44a2-8e60-650111de5909 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:39.343209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:39.344079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:39.344891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:39.344891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:39.344891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:39.344891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:39.344891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:39.344891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:39.344891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:39.344891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:39.344891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:39.344891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:39.344891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:36:39.344891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:39.346059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a44012-7d68-41f7-9245-5ec2870cefc4 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:39.347906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 176/363] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:39.351219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086fd69-193e-44a2-8e60-650111de5909 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:39.358738 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 188/364] 10.222.0.86 () {56 vars in 1376 bytes} [Fri Aug 5 19:36:39 2022] GET /identity/v3/role_assignments?group.id=f5504a89d5b343d7bf5dac61e3410018&scope.project.id=11d6ae35861149a9a30b9eadb5380b85&role.id=686764f0686e446f884315081e158bf3 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:39.359865 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a44012-7d68-41f7-9245-5ec2870cefc4 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:39.517741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a44012-7d68-41f7-9245-5ec2870cefc4 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:39.528932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086fd69-193e-44a2-8e60-650111de5909 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:39.563154 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402308e-557c-419a-af21-5e97efe9b95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:39.563824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402308e-557c-419a-af21-5e97efe9b95a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:39.564104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402308e-557c-419a-af21-5e97efe9b95a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:39.564378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402308e-557c-419a-af21-5e97efe9b95a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:39.577847 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402308e-557c-419a-af21-5e97efe9b95a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:39.583338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22539c62-99f8-461f-b4c5-bf7586b461f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-d402308e-557c-419a-af21-5e97efe9b95a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:39.583917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:39.584769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:39.585565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:39.585565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:39.585565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:39.585565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:39.585565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:39.585565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:39.585565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:39.585565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:39.585565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:39.585565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:39.585565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:39.585565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:39.586171 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 189/365] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:36:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:39.586741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402308e-557c-419a-af21-5e97efe9b95a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:39.587576 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539c62-99f8-461f-b4c5-bf7586b461f0 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:39.588127 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539c62-99f8-461f-b4c5-bf7586b461f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:39.588600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539c62-99f8-461f-b4c5-bf7586b461f0 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:39.589552 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22539c62-99f8-461f-b4c5-bf7586b461f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:39.590225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22539c62-99f8-461f-b4c5-bf7586b461f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:39.591928 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22539c62-99f8-461f-b4c5-bf7586b461f0 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:39.594431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:39.609192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 177/366] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:36:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:39.610688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22539c62-99f8-461f-b4c5-bf7586b461f0 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:39.650259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:39.720487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402308e-557c-419a-af21-5e97efe9b95a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:39.748496 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22539c62-99f8-461f-b4c5-bf7586b461f0 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:39.771144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6814f5b-b4c0-44aa-81cc-563e675731d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:39.772253 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6814f5b-b4c0-44aa-81cc-563e675731d3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:39.772742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6814f5b-b4c0-44aa-81cc-563e675731d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:39.773294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6814f5b-b4c0-44aa-81cc-563e675731d3 None admin] PATH_INFO: `/v3/groups/f5504a89d5b343d7bf5dac61e3410018` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:39.774834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6814f5b-b4c0-44aa-81cc-563e675731d3 None admin] RBAC: Authorizing `identity:get_group(group_id=f5504a89d5b343d7bf5dac61e3410018)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:39.776544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6814f5b-b4c0-44aa-81cc-563e675731d3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:39.778499 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 190/367] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:39 2022] GET /identity/v3/groups/f5504a89d5b343d7bf5dac61e3410018 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:39.779509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6814f5b-b4c0-44aa-81cc-563e675731d3 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:39.801055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a912898-ca00-4d95-acda-e9b23ee55d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:39.801710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a912898-ca00-4d95-acda-e9b23ee55d55 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:39.801991 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a912898-ca00-4d95-acda-e9b23ee55d55 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:39.802264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a912898-ca00-4d95-acda-e9b23ee55d55 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:39.803039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a912898-ca00-4d95-acda-e9b23ee55d55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:39.803491 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a912898-ca00-4d95-acda-e9b23ee55d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:39.806573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a912898-ca00-4d95-acda-e9b23ee55d55 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:39.820043 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 178/368] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:39.820921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a912898-ca00-4d95-acda-e9b23ee55d55 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:39.866017 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:39.902155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6814f5b-b4c0-44aa-81cc-563e675731d3 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:39.941537 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddceb4-a6d2-479e-9724-2bb37812bfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:39.942559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddceb4-a6d2-479e-9724-2bb37812bfe4 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:39.943088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddceb4-a6d2-479e-9724-2bb37812bfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:39.943597 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddceb4-a6d2-479e-9724-2bb37812bfe4 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:39.944530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddceb4-a6d2-479e-9724-2bb37812bfe4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:39.945767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddceb4-a6d2-479e-9724-2bb37812bfe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:39.947778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddceb4-a6d2-479e-9724-2bb37812bfe4 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:39.948846 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:39.959105 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a912898-ca00-4d95-acda-e9b23ee55d55 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:39.961171 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 191/369] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:39.962305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddceb4-a6d2-479e-9724-2bb37812bfe4 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:40.006926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261637c-975c-4b42-b063-345182ee4c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:40.007781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261637c-975c-4b42-b063-345182ee4c3e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:40.007968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261637c-975c-4b42-b063-345182ee4c3e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:40.008240 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4261637c-975c-4b42-b063-345182ee4c3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:40.089173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddceb4-a6d2-479e-9724-2bb37812bfe4 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:40.099569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261637c-975c-4b42-b063-345182ee4c3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:40.100856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4261637c-975c-4b42-b063-345182ee4c3e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:40.127304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdfc7a-79c6-49ca-8678-4b753acad663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:40.128282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdfc7a-79c6-49ca-8678-4b753acad663 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:40.128765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdfc7a-79c6-49ca-8678-4b753acad663 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:40.129237 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdfc7a-79c6-49ca-8678-4b753acad663 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:40.130269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdfc7a-79c6-49ca-8678-4b753acad663 None admin] RBAC: Adding query filter params (group.id=f5504a89d5b343d7bf5dac61e3410018, role.id=fe6c7625134a449c8395ab84ffd82371, scope.project.id=5c33c6fa415942299549b8fdce48c1b7) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:40.130827 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdfc7a-79c6-49ca-8678-4b753acad663 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:40.133182 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdfc7a-79c6-49ca-8678-4b753acad663 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:40.135795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:40.142392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 192/370] 10.222.0.86 () {56 vars in 1376 bytes} [Fri Aug 5 19:36:39 2022] GET /identity/v3/role_assignments?group.id=f5504a89d5b343d7bf5dac61e3410018&scope.project.id=5c33c6fa415942299549b8fdce48c1b7&role.id=fe6c7625134a449c8395ab84ffd82371 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:40.143763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdfc7a-79c6-49ca-8678-4b753acad663 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:40.318779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdfc7a-79c6-49ca-8678-4b753acad663 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:40.322092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:40.347538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 179/371] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:40.350019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261637c-975c-4b42-b063-345182ee4c3e None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:40.403460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d6bbb-fdda-4430-a6eb-4916428a9830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:40.404593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d6bbb-fdda-4430-a6eb-4916428a9830 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:40.405175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d6bbb-fdda-4430-a6eb-4916428a9830 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:40.405661 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d6bbb-fdda-4430-a6eb-4916428a9830 None admin] PATH_INFO: `/v3/users/8de27355f5454e90a0472b2c5b63bd1e` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:40.407144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d6bbb-fdda-4430-a6eb-4916428a9830 None admin] RBAC: Authorizing `identity:get_user(user_id=8de27355f5454e90a0472b2c5b63bd1e)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:40.409033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d6bbb-fdda-4430-a6eb-4916428a9830 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:40.411031 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 193/372] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:36:40 2022] GET /identity/v3/users/8de27355f5454e90a0472b2c5b63bd1e => generated 293 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:40.412232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d6bbb-fdda-4430-a6eb-4916428a9830 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:40.486873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4261637c-975c-4b42-b063-345182ee4c3e None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:40.519089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:40.530607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f558f-817d-4ce0-9814-3c496d95cb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:40.531405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f558f-817d-4ce0-9814-3c496d95cb15 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:40.531675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f558f-817d-4ce0-9814-3c496d95cb15 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:40.531940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f558f-817d-4ce0-9814-3c496d95cb15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:40.578664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d6bbb-fdda-4430-a6eb-4916428a9830 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:40.606824 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:40.621700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f558f-817d-4ce0-9814-3c496d95cb15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:40.624183 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f558f-817d-4ce0-9814-3c496d95cb15 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:40.633318 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efdcf2-c56f-4cb2-9486-5c9c77fd5108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:40.633987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efdcf2-c56f-4cb2-9486-5c9c77fd5108 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:40.634265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efdcf2-c56f-4cb2-9486-5c9c77fd5108 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:40.634544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efdcf2-c56f-4cb2-9486-5c9c77fd5108 None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:40.638203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efdcf2-c56f-4cb2-9486-5c9c77fd5108 None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:40.639833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efdcf2-c56f-4cb2-9486-5c9c77fd5108 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:40.641963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 194/373] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:40 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:40.642657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efdcf2-c56f-4cb2-9486-5c9c77fd5108 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:40.738308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 180/374] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:40.739273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f558f-817d-4ce0-9814-3c496d95cb15 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:40.769852 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efdcf2-c56f-4cb2-9486-5c9c77fd5108 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:40.811993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eed314-c2dc-4215-ba9a-852de9d96709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:40.812872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eed314-c2dc-4215-ba9a-852de9d96709 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:40.813451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eed314-c2dc-4215-ba9a-852de9d96709 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:40.813732 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eed314-c2dc-4215-ba9a-852de9d96709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:40.814517 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eed314-c2dc-4215-ba9a-852de9d96709 None admin] RBAC: Adding query filter params (role.id=78132e8886fd4da988a6fe763a388e16, user.id=8de27355f5454e90a0472b2c5b63bd1e, scope.system=all) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:40.814865 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eed314-c2dc-4215-ba9a-852de9d96709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:40.816576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eed314-c2dc-4215-ba9a-852de9d96709 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:40.822445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:40.833265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 195/375] 10.222.0.86 () {56 vars in 1308 bytes} [Fri Aug 5 19:36:40 2022] GET /identity/v3/role_assignments?user.id=8de27355f5454e90a0472b2c5b63bd1e&scope.system=all&role.id=78132e8886fd4da988a6fe763a388e16 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:40.834560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0468dfa-1dbe-4466-95c1-8bcc9635c14c None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:40.835800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0468dfa-1dbe-4466-95c1-8bcc9635c14c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:40.835800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0468dfa-1dbe-4466-95c1-8bcc9635c14c None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:40.838816 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 196/376] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:40.839672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0468dfa-1dbe-4466-95c1-8bcc9635c14c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:40.867198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:40.901139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f558f-817d-4ce0-9814-3c496d95cb15 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:40.956296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515111c-0a5b-4059-8af6-299af916ceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:40.957364 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515111c-0a5b-4059-8af6-299af916ceb4 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:40.957805 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515111c-0a5b-4059-8af6-299af916ceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:40.958245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515111c-0a5b-4059-8af6-299af916ceb4 None admin] PATH_INFO: `/v3/groups/f5504a89d5b343d7bf5dac61e3410018` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:40.960934 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515111c-0a5b-4059-8af6-299af916ceb4 None admin] RBAC: Authorizing `identity:get_group(group_id=f5504a89d5b343d7bf5dac61e3410018)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:40.963187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515111c-0a5b-4059-8af6-299af916ceb4 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:40.967400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 181/377] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:40 2022] GET /identity/v3/groups/f5504a89d5b343d7bf5dac61e3410018 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:40.968821 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f37c05-e91e-4ea5-b8be-e867981c0bc8 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:40.969101 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f37c05-e91e-4ea5-b8be-e867981c0bc8 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:40.969371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f37c05-e91e-4ea5-b8be-e867981c0bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:40.995169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-67f37c05-e91e-4ea5-b8be-e867981c0bc8 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:40.996414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-67f37c05-e91e-4ea5-b8be-e867981c0bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:41.013255 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0468dfa-1dbe-4466-95c1-8bcc9635c14c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:41.086803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d4571-d4cb-4f3c-83d5-d30902d6c342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:41.087963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d4571-d4cb-4f3c-83d5-d30902d6c342 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:41.088409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d4571-d4cb-4f3c-83d5-d30902d6c342 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:41.089076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d4571-d4cb-4f3c-83d5-d30902d6c342 None admin] PATH_INFO: `/v3/projects/11aa9c879322427d8920955a2aa93539` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:41.095891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d4571-d4cb-4f3c-83d5-d30902d6c342 None admin] RBAC: Authorizing `identity:get_project(project_id=11aa9c879322427d8920955a2aa93539)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:41.097383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d4571-d4cb-4f3c-83d5-d30902d6c342 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:41.104702 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 197/378] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:40 2022] GET /identity/v3/projects/11aa9c879322427d8920955a2aa93539 => generated 344 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:41.104886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 182/379] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:41.105766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d4571-d4cb-4f3c-83d5-d30902d6c342 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:41.109845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f37c05-e91e-4ea5-b8be-e867981c0bc8 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:41.221481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:41.254045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:41.270553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f37c05-e91e-4ea5-b8be-e867981c0bc8 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:41.273852 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d4571-d4cb-4f3c-83d5-d30902d6c342 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:41.321683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034dff7-85b5-461c-859e-580b53a82053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:41.322442 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034dff7-85b5-461c-859e-580b53a82053 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:41.323045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034dff7-85b5-461c-859e-580b53a82053 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:41.323045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034dff7-85b5-461c-859e-580b53a82053 None admin] PATH_INFO: `/v3/groups/2b5cd417939b46cda3fee3d3bba700e9` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:41.330299 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034dff7-85b5-461c-859e-580b53a82053 None admin] RBAC: Authorizing `identity:get_group(group_id=2b5cd417939b46cda3fee3d3bba700e9)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:41.330926 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157392c8-b2a3-4e95-87d4-6edef5698648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:41.331796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157392c8-b2a3-4e95-87d4-6edef5698648 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:41.331910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034dff7-85b5-461c-859e-580b53a82053 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:41.332386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157392c8-b2a3-4e95-87d4-6edef5698648 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:41.332867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157392c8-b2a3-4e95-87d4-6edef5698648 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:41.333857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157392c8-b2a3-4e95-87d4-6edef5698648 None admin] RBAC: Adding query filter params (group.id=f5504a89d5b343d7bf5dac61e3410018, role.id=686764f0686e446f884315081e158bf3, scope.project.id=5c33c6fa415942299549b8fdce48c1b7) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:41.333980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 183/380] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:41 2022] GET /identity/v3/groups/2b5cd417939b46cda3fee3d3bba700e9 => generated 229 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:41.334560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157392c8-b2a3-4e95-87d4-6edef5698648 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:41.334814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034dff7-85b5-461c-859e-580b53a82053 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:41.336983 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157392c8-b2a3-4e95-87d4-6edef5698648 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:41.346035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 198/381] 10.222.0.86 () {56 vars in 1376 bytes} [Fri Aug 5 19:36:41 2022] GET /identity/v3/role_assignments?group.id=f5504a89d5b343d7bf5dac61e3410018&scope.project.id=5c33c6fa415942299549b8fdce48c1b7&role.id=686764f0686e446f884315081e158bf3 => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:41.347371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157392c8-b2a3-4e95-87d4-6edef5698648 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:41.478515 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034dff7-85b5-461c-859e-580b53a82053 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:41.487944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157392c8-b2a3-4e95-87d4-6edef5698648 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:41.531082 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:41.535581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ca329-db59-45c6-a09b-c6e5a4712f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:41.538005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ca329-db59-45c6-a09b-c6e5a4712f36 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:41.538992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ca329-db59-45c6-a09b-c6e5a4712f36 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:41.541610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ca329-db59-45c6-a09b-c6e5a4712f36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:41.553168 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724c069-06ef-412b-95e4-73d3a21a6d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:41.555046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724c069-06ef-412b-95e4-73d3a21a6d5f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:41.557637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724c069-06ef-412b-95e4-73d3a21a6d5f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:41.558138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724c069-06ef-412b-95e4-73d3a21a6d5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:41.559410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724c069-06ef-412b-95e4-73d3a21a6d5f None admin] RBAC: Adding query filter params (group.id=2b5cd417939b46cda3fee3d3bba700e9, role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=11aa9c879322427d8920955a2aa93539) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:41.560814 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724c069-06ef-412b-95e4-73d3a21a6d5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:41.565158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724c069-06ef-412b-95e4-73d3a21a6d5f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:41.570609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ca329-db59-45c6-a09b-c6e5a4712f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:41.577840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 199/382] 10.222.0.86 () {56 vars in 1376 bytes} [Fri Aug 5 19:36:41 2022] GET /identity/v3/role_assignments?group.id=2b5cd417939b46cda3fee3d3bba700e9&scope.project.id=11aa9c879322427d8920955a2aa93539&role.id=93eb204498fc49d297bcd1841e00818d => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-f42ca329-db59-45c6-a09b-c6e5a4712f36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:41.580862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:41.581831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:41.582648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:41.582648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:41.582648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:41.582648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:41.582648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:41.582648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:41.582648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:41.582648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:41.582648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:41.582648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:41.582648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:41.582648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:41.586327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 184/383] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:41.596014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724c069-06ef-412b-95e4-73d3a21a6d5f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:41.728970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724c069-06ef-412b-95e4-73d3a21a6d5f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:41.770422 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994c70a-d4f7-4717-9e38-094128bd8c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:41.771071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994c70a-d4f7-4717-9e38-094128bd8c72 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:41.771342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994c70a-d4f7-4717-9e38-094128bd8c72 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:41.771608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994c70a-d4f7-4717-9e38-094128bd8c72 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:41.772377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994c70a-d4f7-4717-9e38-094128bd8c72 None admin] RBAC: Adding query filter params (name=member) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:41.772681 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994c70a-d4f7-4717-9e38-094128bd8c72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:41.774728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994c70a-d4f7-4717-9e38-094128bd8c72 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:41.784749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 200/384] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:41.790594 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ca329-db59-45c6-a09b-c6e5a4712f36 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:41.810867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:41.933345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ca329-db59-45c6-a09b-c6e5a4712f36 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:41.978224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b351b-c21d-4164-8019-0314ea7df4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:41.979429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b351b-c21d-4164-8019-0314ea7df4cd None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:41.979945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b351b-c21d-4164-8019-0314ea7df4cd None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:41.980432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b351b-c21d-4164-8019-0314ea7df4cd None admin] PATH_INFO: `/v3/groups/f5504a89d5b343d7bf5dac61e3410018` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:41.985274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b351b-c21d-4164-8019-0314ea7df4cd None admin] RBAC: Authorizing `identity:get_group(group_id=f5504a89d5b343d7bf5dac61e3410018)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:41.987175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b351b-c21d-4164-8019-0314ea7df4cd None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:41.989439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 185/385] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:41 2022] GET /identity/v3/groups/f5504a89d5b343d7bf5dac61e3410018 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:41.994955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994c70a-d4f7-4717-9e38-094128bd8c72 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:42.067545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:42.131036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994c70a-d4f7-4717-9e38-094128bd8c72 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:42.170516 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2f2d2-ccd9-4a61-a16f-8397a55f2078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:42.171190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2f2d2-ccd9-4a61-a16f-8397a55f2078 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:42.171634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2f2d2-ccd9-4a61-a16f-8397a55f2078 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:42.171909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2f2d2-ccd9-4a61-a16f-8397a55f2078 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:42.173166 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2f2d2-ccd9-4a61-a16f-8397a55f2078 None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:42.174602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2f2d2-ccd9-4a61-a16f-8397a55f2078 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:42.176295 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 201/386] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:41 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:42.183860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b351b-c21d-4164-8019-0314ea7df4cd None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:42.223815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa89e2e-9c64-49a5-9a63-c6fd04392fec None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:42.224378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa89e2e-9c64-49a5-9a63-c6fd04392fec None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:42.224378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa89e2e-9c64-49a5-9a63-c6fd04392fec None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:42.225685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 202/387] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:42.232340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23d403-3557-4a1c-8366-1a2ac0147860 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:42.234152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23d403-3557-4a1c-8366-1a2ac0147860 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:42.234152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23d403-3557-4a1c-8366-1a2ac0147860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:42.256686 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-5c23d403-3557-4a1c-8366-1a2ac0147860 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:42.259209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5c23d403-3557-4a1c-8366-1a2ac0147860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:42.350482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:42.374912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 203/388] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:42.385448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b351b-c21d-4164-8019-0314ea7df4cd None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:42.422792 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23d403-3557-4a1c-8366-1a2ac0147860 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:42.459456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e79cd7-99ad-4083-87a6-c11ba4a663ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:42.461034 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e79cd7-99ad-4083-87a6-c11ba4a663ca None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:42.461524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e79cd7-99ad-4083-87a6-c11ba4a663ca None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:42.462006 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e79cd7-99ad-4083-87a6-c11ba4a663ca None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85/groups/f5504a89d5b343d7bf5dac61e3410018/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:42.467341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e79cd7-99ad-4083-87a6-c11ba4a663ca None admin] RBAC: Authorizing `identity:create_grant(project_id=11d6ae35861149a9a30b9eadb5380b85, group_id=f5504a89d5b343d7bf5dac61e3410018, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:42.468813 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e79cd7-99ad-4083-87a6-c11ba4a663ca None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:42.505245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 186/389] 10.222.0.86 () {58 vars in 1442 bytes} [Fri Aug 5 19:36:42 2022] PUT /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85/groups/f5504a89d5b343d7bf5dac61e3410018/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:36:42.526945 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:42.587603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23d403-3557-4a1c-8366-1a2ac0147860 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:42.630189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791b049-7c64-46bc-a0d6-56cc1259df51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:42.630873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b049-7c64-46bc-a0d6-56cc1259df51 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:42.631151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b049-7c64-46bc-a0d6-56cc1259df51 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:42.631419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b049-7c64-46bc-a0d6-56cc1259df51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:42.644642 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791b049-7c64-46bc-a0d6-56cc1259df51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-f791b049-7c64-46bc-a0d6-56cc1259df51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:42.650177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:42.652795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:42.653854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:42.653854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:42.653854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:42.653854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:42.653854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:42.653854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:42.653854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:42.653854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:42.653854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:42.653854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:42.653854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:42.653854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:42.653854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 204/390] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:36:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:42.660638 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e79cd7-99ad-4083-87a6-c11ba4a663ca None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:42.805523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e79cd7-99ad-4083-87a6-c11ba4a663ca None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:42.854800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cbc72-05bd-4a05-9a48-555a13a41c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:42.855437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cbc72-05bd-4a05-9a48-555a13a41c2d None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:42.855719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cbc72-05bd-4a05-9a48-555a13a41c2d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:42.855994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cbc72-05bd-4a05-9a48-555a13a41c2d None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:42.856721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cbc72-05bd-4a05-9a48-555a13a41c2d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:42.857027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cbc72-05bd-4a05-9a48-555a13a41c2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:42.858414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cbc72-05bd-4a05-9a48-555a13a41c2d None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:42.870339 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 187/391] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:36:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:42.875515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791b049-7c64-46bc-a0d6-56cc1259df51 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:43.020893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791b049-7c64-46bc-a0d6-56cc1259df51 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:43.034985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:43.070394 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e37bf0-7463-40e8-a5d0-2a9322250f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:43.071460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e37bf0-7463-40e8-a5d0-2a9322250f51 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:43.071956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e37bf0-7463-40e8-a5d0-2a9322250f51 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:43.072430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e37bf0-7463-40e8-a5d0-2a9322250f51 None admin] PATH_INFO: `/v3/groups/f5504a89d5b343d7bf5dac61e3410018` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:43.073904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e37bf0-7463-40e8-a5d0-2a9322250f51 None admin] RBAC: Authorizing `identity:get_group(group_id=f5504a89d5b343d7bf5dac61e3410018)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:43.075587 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e37bf0-7463-40e8-a5d0-2a9322250f51 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:43.078398 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 205/392] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:42 2022] GET /identity/v3/groups/f5504a89d5b343d7bf5dac61e3410018 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:43.083412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cbc72-05bd-4a05-9a48-555a13a41c2d None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:43.088952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:43.234012 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299ccbd-3fdc-42c3-8f60-0ec0001bc31a None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:43.234533 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299ccbd-3fdc-42c3-8f60-0ec0001bc31a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:43.234669 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cbc72-05bd-4a05-9a48-555a13a41c2d None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:43.234767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299ccbd-3fdc-42c3-8f60-0ec0001bc31a None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:43.240032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 206/393] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:43.244125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57620edc-593a-4e82-9e24-57379d39a01d None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:43.244407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57620edc-593a-4e82-9e24-57379d39a01d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:43.244682 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57620edc-593a-4e82-9e24-57379d39a01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:43.271654 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858189a-b4f8-4199-9994-2f62fa843917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:43.272308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858189a-b4f8-4199-9994-2f62fa843917 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:43.272641 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858189a-b4f8-4199-9994-2f62fa843917 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:43.273640 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858189a-b4f8-4199-9994-2f62fa843917 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:43.275770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858189a-b4f8-4199-9994-2f62fa843917 None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:43.277314 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858189a-b4f8-4199-9994-2f62fa843917 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:43.279577 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 188/394] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:43 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:43.283729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-57620edc-593a-4e82-9e24-57379d39a01d None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:43.284945 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-57620edc-593a-4e82-9e24-57379d39a01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:43.287875 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858189a-b4f8-4199-9994-2f62fa843917 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:43.425494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 207/395] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:43.439524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:43.479311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57620edc-593a-4e82-9e24-57379d39a01d None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:43.498250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858189a-b4f8-4199-9994-2f62fa843917 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:43.573459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff316f-88fc-4073-8433-47bfbc62f4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:43.574141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff316f-88fc-4073-8433-47bfbc62f4b4 None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:43.574421 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff316f-88fc-4073-8433-47bfbc62f4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:43.574727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff316f-88fc-4073-8433-47bfbc62f4b4 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85/groups/f5504a89d5b343d7bf5dac61e3410018/roles/686764f0686e446f884315081e158bf3` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:43.577068 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff316f-88fc-4073-8433-47bfbc62f4b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=11d6ae35861149a9a30b9eadb5380b85, group_id=f5504a89d5b343d7bf5dac61e3410018, role_id=686764f0686e446f884315081e158bf3)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:43.578512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff316f-88fc-4073-8433-47bfbc62f4b4 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:43.610076 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 189/396] 10.222.0.86 () {58 vars in 1442 bytes} [Fri Aug 5 19:36:43 2022] PUT /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85/groups/f5504a89d5b343d7bf5dac61e3410018/roles/686764f0686e446f884315081e158bf3 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:36:43.671400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57620edc-593a-4e82-9e24-57379d39a01d None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:43.686667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:43.736323 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f553b59-41b9-46af-a89e-8d8d8560c5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:43.737402 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f553b59-41b9-46af-a89e-8d8d8560c5e6 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:43.737862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f553b59-41b9-46af-a89e-8d8d8560c5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:43.738273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f553b59-41b9-46af-a89e-8d8d8560c5e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:43.756529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f553b59-41b9-46af-a89e-8d8d8560c5e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-9f553b59-41b9-46af-a89e-8d8d8560c5e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:43.767159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:43.768139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:43.768938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:43.768938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:43.768938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:43.768938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:43.768938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:43.768938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:43.768938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:43.768938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:43.768938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:43.768938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:43.768938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:43.768938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:43.774791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 208/397] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:43.779637 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff316f-88fc-4073-8433-47bfbc62f4b4 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:43.946749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff316f-88fc-4073-8433-47bfbc62f4b4 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:43.979289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:43.985564 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559e470-e01c-4fdc-ae7a-1d7569689f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:43.986716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559e470-e01c-4fdc-ae7a-1d7569689f57 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:43.987607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559e470-e01c-4fdc-ae7a-1d7569689f57 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:43.987850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559e470-e01c-4fdc-ae7a-1d7569689f57 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:43.988584 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559e470-e01c-4fdc-ae7a-1d7569689f57 None admin] RBAC: Adding query filter params (name=member) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:43.988894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559e470-e01c-4fdc-ae7a-1d7569689f57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:43.990311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559e470-e01c-4fdc-ae7a-1d7569689f57 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:44.012640 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 190/398] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:44.014054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f553b59-41b9-46af-a89e-8d8d8560c5e6 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:44.128114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f553b59-41b9-46af-a89e-8d8d8560c5e6 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:44.165784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd26507-6e99-43eb-b987-9242c5a812e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:44.166430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd26507-6e99-43eb-b987-9242c5a812e5 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:44.166747 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd26507-6e99-43eb-b987-9242c5a812e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:44.167022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd26507-6e99-43eb-b987-9242c5a812e5 None admin] PATH_INFO: `/v3/groups/f5504a89d5b343d7bf5dac61e3410018` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:44.168231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd26507-6e99-43eb-b987-9242c5a812e5 None admin] RBAC: Authorizing `identity:get_group(group_id=f5504a89d5b343d7bf5dac61e3410018)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:44.169760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd26507-6e99-43eb-b987-9242c5a812e5 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:44.176187 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 209/399] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:44 2022] GET /identity/v3/groups/f5504a89d5b343d7bf5dac61e3410018 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:44.177653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559e470-e01c-4fdc-ae7a-1d7569689f57 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:44.298271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559e470-e01c-4fdc-ae7a-1d7569689f57 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:44.344028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a01170-cd9f-4a36-96ea-0bc114abd891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:44.344777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a01170-cd9f-4a36-96ea-0bc114abd891 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:44.345063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a01170-cd9f-4a36-96ea-0bc114abd891 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:44.345333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a01170-cd9f-4a36-96ea-0bc114abd891 None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:44.352382 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a01170-cd9f-4a36-96ea-0bc114abd891 None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:44.354612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a01170-cd9f-4a36-96ea-0bc114abd891 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:44.356693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 191/400] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:44 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:44.362932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd26507-6e99-43eb-b987-9242c5a812e5 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:44.419583 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:44.422198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23d319-13bb-4323-abb1-967a0bf33d72 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:44.422809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23d319-13bb-4323-abb1-967a0bf33d72 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:44.423296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23d319-13bb-4323-abb1-967a0bf33d72 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:44.425180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 192/401] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:44.433998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7115ea8-b19c-4dd9-8852-1fc76346ddb9 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:44.434306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7115ea8-b19c-4dd9-8852-1fc76346ddb9 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:44.434546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7115ea8-b19c-4dd9-8852-1fc76346ddb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:44.455405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-b7115ea8-b19c-4dd9-8852-1fc76346ddb9 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:44.456621 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b7115ea8-b19c-4dd9-8852-1fc76346ddb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:44.527021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:44.546734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 193/402] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:44.553517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b338b-8739-41c5-8f64-11c525e57a72 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:44.553735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b338b-8739-41c5-8f64-11c525e57a72 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:44.554574 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b338b-8739-41c5-8f64-11c525e57a72 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:44.557704 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 194/403] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:44.565134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd26507-6e99-43eb-b987-9242c5a812e5 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:44.569388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fe12f-3e4e-4de8-9bab-95135dc3eb98 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:44.570305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fe12f-3e4e-4de8-9bab-95135dc3eb98 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:44.571427 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9fe12f-3e4e-4de8-9bab-95135dc3eb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:44.597486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-9a9fe12f-3e4e-4de8-9bab-95135dc3eb98 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:44.598712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9a9fe12f-3e4e-4de8-9bab-95135dc3eb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:44.612593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11ad23-dfc8-4b46-addb-7c913e034342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:44.613369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11ad23-dfc8-4b46-addb-7c913e034342 None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:44.613717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11ad23-dfc8-4b46-addb-7c913e034342 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:44.614079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11ad23-dfc8-4b46-addb-7c913e034342 None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7/groups/f5504a89d5b343d7bf5dac61e3410018/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:44.616340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11ad23-dfc8-4b46-addb-7c913e034342 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c33c6fa415942299549b8fdce48c1b7, group_id=f5504a89d5b343d7bf5dac61e3410018, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:44.619228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11ad23-dfc8-4b46-addb-7c913e034342 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:44.639386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 210/404] 10.222.0.86 () {58 vars in 1442 bytes} [Fri Aug 5 19:36:44 2022] PUT /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7/groups/f5504a89d5b343d7bf5dac61e3410018/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:36:44.640736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11ad23-dfc8-4b46-addb-7c913e034342 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:44.708940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 195/405] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:44.752420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fe12f-3e4e-4de8-9bab-95135dc3eb98 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:44.815802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:44.828390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11ad23-dfc8-4b46-addb-7c913e034342 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:44.873755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01d30e-16d6-4946-89d7-d95a2e72a77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:44.874415 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01d30e-16d6-4946-89d7-d95a2e72a77a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:44.874727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01d30e-16d6-4946-89d7-d95a2e72a77a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:44.875002 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01d30e-16d6-4946-89d7-d95a2e72a77a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:44.889420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01d30e-16d6-4946-89d7-d95a2e72a77a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-1f01d30e-16d6-4946-89d7-d95a2e72a77a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:44.896728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:44.897836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:44.899199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:44.899199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:44.899199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:44.899199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:44.899199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:44.899199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:44.899199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:44.899199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:44.899199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:44.899199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:44.899199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:44.899199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:44.899199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 211/406] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:36:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:44.907318 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01d30e-16d6-4946-89d7-d95a2e72a77a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:44.972239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9fe12f-3e4e-4de8-9bab-95135dc3eb98 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:44.995701 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:45.013118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6e890-bdad-45bb-b187-b229c3d58c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:45.014048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6e890-bdad-45bb-b187-b229c3d58c94 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:45.014489 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6e890-bdad-45bb-b187-b229c3d58c94 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:45.015066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6e890-bdad-45bb-b187-b229c3d58c94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:45.028289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6e890-bdad-45bb-b187-b229c3d58c94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-09f6e890-bdad-45bb-b187-b229c3d58c94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:45.034963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:45.036055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:45.037078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:45.037078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:45.037078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:45.037078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:45.037078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:45.037078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:45.037078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:45.037078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:45.037078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:45.037078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:45.037078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:45.037078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:45.039718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 196/407] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:36:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:45.049422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6e890-bdad-45bb-b187-b229c3d58c94 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:45.070766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01d30e-16d6-4946-89d7-d95a2e72a77a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:45.121117 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f8882-554a-4195-99d5-7f6451ae4dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:45.121611 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f8882-554a-4195-99d5-7f6451ae4dc0 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:45.121891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f8882-554a-4195-99d5-7f6451ae4dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:45.122162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f8882-554a-4195-99d5-7f6451ae4dc0 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:45.122918 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f8882-554a-4195-99d5-7f6451ae4dc0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:45.123240 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f8882-554a-4195-99d5-7f6451ae4dc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:45.124644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f8882-554a-4195-99d5-7f6451ae4dc0 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:45.134680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 212/408] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:36:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:45.141830 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f8882-554a-4195-99d5-7f6451ae4dc0 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:45.179760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:45.256372 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6e890-bdad-45bb-b187-b229c3d58c94 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:45.299958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c7528-55ac-4fa2-a957-189adebcd0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:45.301092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c7528-55ac-4fa2-a957-189adebcd0c9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:45.301525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c7528-55ac-4fa2-a957-189adebcd0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:45.301941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c7528-55ac-4fa2-a957-189adebcd0c9 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:45.303162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c7528-55ac-4fa2-a957-189adebcd0c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:45.303625 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c7528-55ac-4fa2-a957-189adebcd0c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:45.306500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c7528-55ac-4fa2-a957-189adebcd0c9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:45.320926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 197/409] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:45.329287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c7528-55ac-4fa2-a957-189adebcd0c9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:45.351545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f8882-554a-4195-99d5-7f6451ae4dc0 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:45.416685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:45.447381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd989b0-cd57-464d-a3dd-794dc77f3cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:45.448083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd989b0-cd57-464d-a3dd-794dc77f3cf1 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:45.448555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd989b0-cd57-464d-a3dd-794dc77f3cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:45.448881 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd989b0-cd57-464d-a3dd-794dc77f3cf1 None admin] PATH_INFO: `/v3/groups/f5504a89d5b343d7bf5dac61e3410018` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:45.450214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd989b0-cd57-464d-a3dd-794dc77f3cf1 None admin] RBAC: Authorizing `identity:get_group(group_id=f5504a89d5b343d7bf5dac61e3410018)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:45.451949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd989b0-cd57-464d-a3dd-794dc77f3cf1 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:45.464116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 213/410] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:45 2022] GET /identity/v3/groups/f5504a89d5b343d7bf5dac61e3410018 => generated 226 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:45.470998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd989b0-cd57-464d-a3dd-794dc77f3cf1 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:45.499151 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c7528-55ac-4fa2-a957-189adebcd0c9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:45.549522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2689f-5673-4684-aae2-4b272e5a234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:45.550279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2689f-5673-4684-aae2-4b272e5a234d None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:45.550559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2689f-5673-4684-aae2-4b272e5a234d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:45.550875 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2689f-5673-4684-aae2-4b272e5a234d None admin] PATH_INFO: `/v3/groups/2b5cd417939b46cda3fee3d3bba700e9` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:45.552319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2689f-5673-4684-aae2-4b272e5a234d None admin] RBAC: Authorizing `identity:get_group(group_id=2b5cd417939b46cda3fee3d3bba700e9)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:45.554722 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2689f-5673-4684-aae2-4b272e5a234d None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:45.558777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 198/411] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:45 2022] GET /identity/v3/groups/2b5cd417939b46cda3fee3d3bba700e9 => generated 229 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:45.564013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2689f-5673-4684-aae2-4b272e5a234d None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:45.670161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:45.709408 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd989b0-cd57-464d-a3dd-794dc77f3cf1 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:45.720180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2689f-5673-4684-aae2-4b272e5a234d None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:45.755212 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:45.786782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863cd23-5c2d-4c07-9d1c-afecaf30b4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:45.787565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49f56d-dcf2-41d0-b08d-d247c5d7abe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:45.787698 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863cd23-5c2d-4c07-9d1c-afecaf30b4a6 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:45.787986 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863cd23-5c2d-4c07-9d1c-afecaf30b4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:45.788206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49f56d-dcf2-41d0-b08d-d247c5d7abe1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:45.788997 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863cd23-5c2d-4c07-9d1c-afecaf30b4a6 None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:45.789116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49f56d-dcf2-41d0-b08d-d247c5d7abe1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:45.789317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49f56d-dcf2-41d0-b08d-d247c5d7abe1 None admin] PATH_INFO: `/v3/projects/11aa9c879322427d8920955a2aa93539` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:45.790513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49f56d-dcf2-41d0-b08d-d247c5d7abe1 None admin] RBAC: Authorizing `identity:get_project(project_id=11aa9c879322427d8920955a2aa93539)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:45.790648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863cd23-5c2d-4c07-9d1c-afecaf30b4a6 None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:45.792111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863cd23-5c2d-4c07-9d1c-afecaf30b4a6 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:45.792391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49f56d-dcf2-41d0-b08d-d247c5d7abe1 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:45.794498 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 199/412] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:45 2022] GET /identity/v3/projects/11aa9c879322427d8920955a2aa93539 => generated 344 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:45.795222 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 214/413] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:45 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:45.799660 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4ca59-c1ac-4a0a-991c-b9596b837805 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:45.799949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4ca59-c1ac-4a0a-991c-b9596b837805 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:45.800042 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863cd23-5c2d-4c07-9d1c-afecaf30b4a6 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:45.800225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4ca59-c1ac-4a0a-991c-b9596b837805 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:45.806662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 200/414] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:45.807582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4ca59-c1ac-4a0a-991c-b9596b837805 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:45.932486 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863cd23-5c2d-4c07-9d1c-afecaf30b4a6 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:45.933312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4ca59-c1ac-4a0a-991c-b9596b837805 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:45.971962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b24fae-3f81-4c38-9dd4-5b4e89f1ed02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:45.972652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b24fae-3f81-4c38-9dd4-5b4e89f1ed02 None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:45.972932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b24fae-3f81-4c38-9dd4-5b4e89f1ed02 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:45.973198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b24fae-3f81-4c38-9dd4-5b4e89f1ed02 None admin] PATH_INFO: `/v3/projects/11aa9c879322427d8920955a2aa93539/groups/2b5cd417939b46cda3fee3d3bba700e9/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:45.975461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b24fae-3f81-4c38-9dd4-5b4e89f1ed02 None admin] RBAC: Authorizing `identity:create_grant(project_id=11aa9c879322427d8920955a2aa93539, group_id=2b5cd417939b46cda3fee3d3bba700e9, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:45.977176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b24fae-3f81-4c38-9dd4-5b4e89f1ed02 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:45.984178 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4219e-00fe-4f4e-afa4-3de0dbbd82ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:45.984876 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4219e-00fe-4f4e-afa4-3de0dbbd82ec None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:45.985286 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4219e-00fe-4f4e-afa4-3de0dbbd82ec None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:45.985493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4219e-00fe-4f4e-afa4-3de0dbbd82ec None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7/groups/f5504a89d5b343d7bf5dac61e3410018/roles/686764f0686e446f884315081e158bf3` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:45.987844 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4219e-00fe-4f4e-afa4-3de0dbbd82ec None admin] RBAC: Authorizing `identity:create_grant(project_id=5c33c6fa415942299549b8fdce48c1b7, group_id=f5504a89d5b343d7bf5dac61e3410018, role_id=686764f0686e446f884315081e158bf3)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:45.989276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4219e-00fe-4f4e-afa4-3de0dbbd82ec None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:46.007238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 215/415] 10.222.0.86 () {58 vars in 1442 bytes} [Fri Aug 5 19:36:45 2022] PUT /identity/v3/projects/11aa9c879322427d8920955a2aa93539/groups/2b5cd417939b46cda3fee3d3bba700e9/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:36:46.014692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2760898-e5ad-4e0a-8cb9-07110b41ee39 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:46.014692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2760898-e5ad-4e0a-8cb9-07110b41ee39 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:46.014692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2760898-e5ad-4e0a-8cb9-07110b41ee39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:46.021195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 201/416] 10.222.0.86 () {58 vars in 1442 bytes} [Fri Aug 5 19:36:45 2022] PUT /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7/groups/f5504a89d5b343d7bf5dac61e3410018/roles/686764f0686e446f884315081e158bf3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:36:46.053213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-a2760898-e5ad-4e0a-8cb9-07110b41ee39 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:46.055809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a2760898-e5ad-4e0a-8cb9-07110b41ee39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:46.070717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:46.162152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 216/417] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:46.208408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4219e-00fe-4f4e-afa4-3de0dbbd82ec None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:46.264513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:46.348524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4219e-00fe-4f4e-afa4-3de0dbbd82ec None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:46.393723 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c424b-97ac-484c-9489-11e0a3d16b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:46.394449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c424b-97ac-484c-9489-11e0a3d16b44 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:46.394759 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c424b-97ac-484c-9489-11e0a3d16b44 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:46.395034 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c424b-97ac-484c-9489-11e0a3d16b44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:46.409256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c424b-97ac-484c-9489-11e0a3d16b44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-af4c424b-97ac-484c-9489-11e0a3d16b44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:46.415618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:46.416557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:46.417457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:46.417457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:46.417457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:46.417457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:46.417457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:46.417457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:46.417457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:46.417457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:46.417457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:46.417457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:46.417457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:46.417457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:46.417457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 202/418] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:46.423209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2760898-e5ad-4e0a-8cb9-07110b41ee39 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:46.434691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b186ada-39d0-43bd-b0f2-012bfb92cffc None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:46.434691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b186ada-39d0-43bd-b0f2-012bfb92cffc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:46.434691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b186ada-39d0-43bd-b0f2-012bfb92cffc None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:46.435938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 203/419] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:36:46.443000 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679f1ae-f775-4d77-9f38-5b5043d2ec08 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:46.443957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679f1ae-f775-4d77-9f38-5b5043d2ec08 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:46.444430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679f1ae-f775-4d77-9f38-5b5043d2ec08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:46.466521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-4679f1ae-f775-4d77-9f38-5b5043d2ec08 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:46.468660 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-4679f1ae-f775-4d77-9f38-5b5043d2ec08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:46.557436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 204/420] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:46.590112 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679f1ae-f775-4d77-9f38-5b5043d2ec08 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:46.653208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:46.689882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2760898-e5ad-4e0a-8cb9-07110b41ee39 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:46.751573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679f1ae-f775-4d77-9f38-5b5043d2ec08 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:46.770586 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde9414-8333-4073-8d3d-c47d775a82ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:46.771341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde9414-8333-4073-8d3d-c47d775a82ae None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:46.771620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde9414-8333-4073-8d3d-c47d775a82ae None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:46.771894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde9414-8333-4073-8d3d-c47d775a82ae None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:46.772652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde9414-8333-4073-8d3d-c47d775a82ae None admin] RBAC: Adding query filter params (name=member) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:46.773661 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde9414-8333-4073-8d3d-c47d775a82ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:46.775287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde9414-8333-4073-8d3d-c47d775a82ae None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:46.790029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904a74e-fa2c-4936-a23d-15486d6aba85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:46.790809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904a74e-fa2c-4936-a23d-15486d6aba85 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:46.791150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904a74e-fa2c-4936-a23d-15486d6aba85 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:46.791464 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904a74e-fa2c-4936-a23d-15486d6aba85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:46.796957 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 217/421] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:46.797906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde9414-8333-4073-8d3d-c47d775a82ae None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:46.801880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:46.810676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904a74e-fa2c-4936-a23d-15486d6aba85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-a904a74e-fa2c-4936-a23d-15486d6aba85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:46.816082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:46.816972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:46.817787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:46.817787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:46.817787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:46.817787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:46.817787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:46.817787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:46.817787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:46.817787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:46.817787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:46.817787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:46.817787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:46.817787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:46.817787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 205/422] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:36:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:46.823154 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904a74e-fa2c-4936-a23d-15486d6aba85 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:46.991730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde9414-8333-4073-8d3d-c47d775a82ae None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:47.005896 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:47.022009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904a74e-fa2c-4936-a23d-15486d6aba85 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:47.047284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f64df-fc88-43a7-b33d-241fab43ff0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:47.047911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f64df-fc88-43a7-b33d-241fab43ff0a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:47.048192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f64df-fc88-43a7-b33d-241fab43ff0a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:47.048470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f64df-fc88-43a7-b33d-241fab43ff0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:47.083967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c942d78-905a-4a9f-b847-b11489a477e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:47.085076 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c942d78-905a-4a9f-b847-b11489a477e1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:47.086686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c942d78-905a-4a9f-b847-b11489a477e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:47.087154 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c942d78-905a-4a9f-b847-b11489a477e1 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:47.088228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c942d78-905a-4a9f-b847-b11489a477e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:47.088887 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c942d78-905a-4a9f-b847-b11489a477e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:47.090693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c942d78-905a-4a9f-b847-b11489a477e1 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:47.101863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 206/423] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:36:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:47.107525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c942d78-905a-4a9f-b847-b11489a477e1 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:47.194026 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f64df-fc88-43a7-b33d-241fab43ff0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:47.196396 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f64df-fc88-43a7-b33d-241fab43ff0a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:47.205778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:47.268767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c942d78-905a-4a9f-b847-b11489a477e1 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:47.311080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 218/424] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:47.312734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7ebcd-b5c5-466e-a188-adf4b59377f8 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:47.313469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7ebcd-b5c5-466e-a188-adf4b59377f8 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:47.313978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7ebcd-b5c5-466e-a188-adf4b59377f8 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:47.315498 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 219/425] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:47.316885 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7ebcd-b5c5-466e-a188-adf4b59377f8 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:47.318306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962d09b-5d0c-487a-a36b-8558db3cf3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:47.319146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962d09b-5d0c-487a-a36b-8558db3cf3f3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:47.320218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962d09b-5d0c-487a-a36b-8558db3cf3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:47.320636 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962d09b-5d0c-487a-a36b-8558db3cf3f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:47.347106 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:47.403970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962d09b-5d0c-487a-a36b-8558db3cf3f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:47.405667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962d09b-5d0c-487a-a36b-8558db3cf3f3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:47.441703 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7ebcd-b5c5-466e-a188-adf4b59377f8 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:47.445054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:47.490895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59bff2a-7148-44d2-bcc7-52fd936fe36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:47.491722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bff2a-7148-44d2-bcc7-52fd936fe36e None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:47.492132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bff2a-7148-44d2-bcc7-52fd936fe36e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:47.492632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59bff2a-7148-44d2-bcc7-52fd936fe36e None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:47.494092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59bff2a-7148-44d2-bcc7-52fd936fe36e None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:47.496034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59bff2a-7148-44d2-bcc7-52fd936fe36e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:47.498237 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 220/426] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:47 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:47.499821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aaca5-3d15-482e-bfae-762389d3ccd8 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:47.500347 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aaca5-3d15-482e-bfae-762389d3ccd8 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:47.500808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aaca5-3d15-482e-bfae-762389d3ccd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:47.522515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-512aaca5-3d15-482e-bfae-762389d3ccd8 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:47.525086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-512aaca5-3d15-482e-bfae-762389d3ccd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:47.529056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 207/427] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:36:47.529647 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962d09b-5d0c-487a-a36b-8558db3cf3f3 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:47.586115 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:47.607398 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 221/428] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:47.608764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512aaca5-3d15-482e-bfae-762389d3ccd8 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:47.643065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962d09b-5d0c-487a-a36b-8558db3cf3f3 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:47.684968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb21a58-61a4-4279-9362-fb0e07413cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:47.685642 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb21a58-61a4-4279-9362-fb0e07413cbb None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:47.685984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb21a58-61a4-4279-9362-fb0e07413cbb None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:47.686258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb21a58-61a4-4279-9362-fb0e07413cbb None admin] PATH_INFO: `/v3/groups/f5504a89d5b343d7bf5dac61e3410018` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:47.687547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb21a58-61a4-4279-9362-fb0e07413cbb None admin] RBAC: Authorizing `identity:get_group(group_id=f5504a89d5b343d7bf5dac61e3410018)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:47.689378 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb21a58-61a4-4279-9362-fb0e07413cbb None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:47.691198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 208/429] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:47 2022] GET /identity/v3/groups/f5504a89d5b343d7bf5dac61e3410018 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:47.696773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb21a58-61a4-4279-9362-fb0e07413cbb None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:47.727543 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512aaca5-3d15-482e-bfae-762389d3ccd8 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:47.730688 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:47.772500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e98800-12f9-4af5-b93f-941cc8a07cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:47.773835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e98800-12f9-4af5-b93f-941cc8a07cf3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:47.774339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e98800-12f9-4af5-b93f-941cc8a07cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:47.774900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e98800-12f9-4af5-b93f-941cc8a07cf3 None admin] PATH_INFO: `/v3/projects/11d6ae35861149a9a30b9eadb5380b85` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:47.776403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e98800-12f9-4af5-b93f-941cc8a07cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=11d6ae35861149a9a30b9eadb5380b85)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:47.778288 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e98800-12f9-4af5-b93f-941cc8a07cf3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:47.780742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 222/430] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:47 2022] GET /identity/v3/projects/11d6ae35861149a9a30b9eadb5380b85 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:47.782019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e98800-12f9-4af5-b93f-941cc8a07cf3 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:47.820100 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb21a58-61a4-4279-9362-fb0e07413cbb None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:47.842073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:47.862088 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d0ee3-6dcf-4eac-a891-0c5021c169fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:47.862740 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d0ee3-6dcf-4eac-a891-0c5021c169fe None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:47.863019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d0ee3-6dcf-4eac-a891-0c5021c169fe None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:47.863290 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d0ee3-6dcf-4eac-a891-0c5021c169fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:47.875472 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d0ee3-6dcf-4eac-a891-0c5021c169fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-519d0ee3-6dcf-4eac-a891-0c5021c169fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:47.880910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:47.881834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:47.882726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:47.882726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:47.882726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:47.882726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:47.882726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:47.882726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:47.882726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:47.882726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:47.882726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:47.882726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:47.882726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:36:47.882726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:47.882726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 209/431] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:47.883873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d0ee3-6dcf-4eac-a891-0c5021c169fe None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:47.909237 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e98800-12f9-4af5-b93f-941cc8a07cf3 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:47.950733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e2779-f169-4c26-bf4c-6a541b67b787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:47.951819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e2779-f169-4c26-bf4c-6a541b67b787 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:47.952399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e2779-f169-4c26-bf4c-6a541b67b787 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:47.952871 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e2779-f169-4c26-bf4c-6a541b67b787 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:47.953917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e2779-f169-4c26-bf4c-6a541b67b787 None admin] RBAC: Adding query filter params (group.id=f5504a89d5b343d7bf5dac61e3410018, role.id=fe6c7625134a449c8395ab84ffd82371, scope.project.id=11d6ae35861149a9a30b9eadb5380b85) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:47.954447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e2779-f169-4c26-bf4c-6a541b67b787 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:47.956303 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e2779-f169-4c26-bf4c-6a541b67b787 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:47.963043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 223/432] 10.222.0.86 () {56 vars in 1376 bytes} [Fri Aug 5 19:36:47 2022] GET /identity/v3/role_assignments?group.id=f5504a89d5b343d7bf5dac61e3410018&scope.project.id=11d6ae35861149a9a30b9eadb5380b85&role.id=fe6c7625134a449c8395ab84ffd82371 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:47.964157 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e2779-f169-4c26-bf4c-6a541b67b787 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:48.011635 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:48.028527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d0ee3-6dcf-4eac-a891-0c5021c169fe None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:48.095570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e2779-f169-4c26-bf4c-6a541b67b787 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:48.104583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a4126-b1b9-40cf-8e5d-2bc689a84b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:48.105245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a4126-b1b9-40cf-8e5d-2bc689a84b22 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:48.105524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a4126-b1b9-40cf-8e5d-2bc689a84b22 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:48.105797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a4126-b1b9-40cf-8e5d-2bc689a84b22 None admin] PATH_INFO: `/v3/groups/f5504a89d5b343d7bf5dac61e3410018` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:48.107327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a4126-b1b9-40cf-8e5d-2bc689a84b22 None admin] RBAC: Authorizing `identity:get_group(group_id=f5504a89d5b343d7bf5dac61e3410018)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:48.108791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a4126-b1b9-40cf-8e5d-2bc689a84b22 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:48.114281 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 210/433] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:47 2022] GET /identity/v3/groups/f5504a89d5b343d7bf5dac61e3410018 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:48.119650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a4126-b1b9-40cf-8e5d-2bc689a84b22 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:48.142800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027792ee-6a15-44f9-a03f-a74948210807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:48.143833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027792ee-6a15-44f9-a03f-a74948210807 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:48.144328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027792ee-6a15-44f9-a03f-a74948210807 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:48.144795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027792ee-6a15-44f9-a03f-a74948210807 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:48.145753 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027792ee-6a15-44f9-a03f-a74948210807 None admin] RBAC: Adding query filter params (name=member) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:48.146271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027792ee-6a15-44f9-a03f-a74948210807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:48.148677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027792ee-6a15-44f9-a03f-a74948210807 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:48.164707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 224/434] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:36:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:48.170188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027792ee-6a15-44f9-a03f-a74948210807 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:48.217161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:48.310757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:48.318173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027792ee-6a15-44f9-a03f-a74948210807 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:48.324744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a4126-b1b9-40cf-8e5d-2bc689a84b22 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:48.380020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d45545-7a79-4c10-a3eb-b26bbb02268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:48.380658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d45545-7a79-4c10-a3eb-b26bbb02268b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:48.381432 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d45545-7a79-4c10-a3eb-b26bbb02268b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:48.381705 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d45545-7a79-4c10-a3eb-b26bbb02268b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:48.392279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe41642-47a8-48c4-afd8-181964e3cf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:48.393047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe41642-47a8-48c4-afd8-181964e3cf83 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:48.393753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe41642-47a8-48c4-afd8-181964e3cf83 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:48.394119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe41642-47a8-48c4-afd8-181964e3cf83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:48.395065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe41642-47a8-48c4-afd8-181964e3cf83 None admin] RBAC: Adding query filter params (group.id=f5504a89d5b343d7bf5dac61e3410018, role.id=686764f0686e446f884315081e158bf3, scope.project.id=11d6ae35861149a9a30b9eadb5380b85) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:48.396550 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe41642-47a8-48c4-afd8-181964e3cf83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:48.400010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe41642-47a8-48c4-afd8-181964e3cf83 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:48.414746 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 211/435] 10.222.0.86 () {56 vars in 1376 bytes} [Fri Aug 5 19:36:48 2022] GET /identity/v3/role_assignments?group.id=f5504a89d5b343d7bf5dac61e3410018&scope.project.id=11d6ae35861149a9a30b9eadb5380b85&role.id=686764f0686e446f884315081e158bf3 => generated 611 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:48.468001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d45545-7a79-4c10-a3eb-b26bbb02268b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:48.469579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d45545-7a79-4c10-a3eb-b26bbb02268b None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:48.583137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:48.595008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 225/436] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:36:48.603842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe41642-47a8-48c4-afd8-181964e3cf83 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:48.662891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce83bf-6c66-401c-9e88-be16e1327709 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:48.663612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce83bf-6c66-401c-9e88-be16e1327709 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:48.663743 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce83bf-6c66-401c-9e88-be16e1327709 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:48.667916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 226/437] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:48.672765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965df7a-edc1-4aaa-ba30-7fc0c2069787 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:48.673362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965df7a-edc1-4aaa-ba30-7fc0c2069787 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:48.673834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965df7a-edc1-4aaa-ba30-7fc0c2069787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:48.703884 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3965df7a-edc1-4aaa-ba30-7fc0c2069787 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:48.705192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3965df7a-edc1-4aaa-ba30-7fc0c2069787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:48.775778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe41642-47a8-48c4-afd8-181964e3cf83 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:48.789573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:48.820609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 227/438] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:48.825835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dcd5a-0b81-41a1-99d7-8135ebe642b8 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:48.826356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dcd5a-0b81-41a1-99d7-8135ebe642b8 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:48.826694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dcd5a-0b81-41a1-99d7-8135ebe642b8 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:48.828262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 228/439] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:48.835704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cb524-8036-438d-812e-8b98cf2168de None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:48.836284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cb524-8036-438d-812e-8b98cf2168de None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:48.837008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cb524-8036-438d-812e-8b98cf2168de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:48.852970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00c236-b2e4-471e-a5cc-b3df4a37562a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:48.853626 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00c236-b2e4-471e-a5cc-b3df4a37562a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:48.853907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00c236-b2e4-471e-a5cc-b3df4a37562a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:48.854179 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00c236-b2e4-471e-a5cc-b3df4a37562a None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:48.855999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00c236-b2e4-471e-a5cc-b3df4a37562a None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:48.858085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00c236-b2e4-471e-a5cc-b3df4a37562a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:48.862221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 212/440] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:48 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:48.862221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00c236-b2e4-471e-a5cc-b3df4a37562a None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:48.869511 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-320cb524-8036-438d-812e-8b98cf2168de None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:48.871966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-320cb524-8036-438d-812e-8b98cf2168de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:48.984660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 229/441] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:48.985525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cb524-8036-438d-812e-8b98cf2168de None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:49.015593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00c236-b2e4-471e-a5cc-b3df4a37562a None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:49.060411 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a201f3-780c-4936-9010-6482af864181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:49.061057 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a201f3-780c-4936-9010-6482af864181 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:49.061345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a201f3-780c-4936-9010-6482af864181 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:49.061655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a201f3-780c-4936-9010-6482af864181 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:49.067672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:49.076807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a201f3-780c-4936-9010-6482af864181 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-17a201f3-780c-4936-9010-6482af864181 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:49.082395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:49.083453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:49.084279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:49.084279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:49.084279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:49.084279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:49.084279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:49.084279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:49.084279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:49.084279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:49.084279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:49.084279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:49.084279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:49.084279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:49.084279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 213/442] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:36:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:49.084922 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a201f3-780c-4936-9010-6482af864181 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:49.124946 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cb524-8036-438d-812e-8b98cf2168de None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:49.137135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:49.168934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a9df6-c757-43f7-a07f-3f7099060e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:49.170951 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a9df6-c757-43f7-a07f-3f7099060e3c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:49.171230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a9df6-c757-43f7-a07f-3f7099060e3c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:49.171499 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a9df6-c757-43f7-a07f-3f7099060e3c None admin] PATH_INFO: `/v3/groups/f5504a89d5b343d7bf5dac61e3410018` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:49.173000 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a9df6-c757-43f7-a07f-3f7099060e3c None admin] RBAC: Authorizing `identity:get_group(group_id=f5504a89d5b343d7bf5dac61e3410018)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:49.174447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a9df6-c757-43f7-a07f-3f7099060e3c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:49.176144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 230/443] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:48 2022] GET /identity/v3/groups/f5504a89d5b343d7bf5dac61e3410018 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:49.176911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a9df6-c757-43f7-a07f-3f7099060e3c None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:49.226818 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a201f3-780c-4936-9010-6482af864181 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:49.265763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fae1b8-f39a-4239-8657-1c51d9dfdf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:49.266395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fae1b8-f39a-4239-8657-1c51d9dfdf6d None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:49.266697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fae1b8-f39a-4239-8657-1c51d9dfdf6d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:49.266966 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fae1b8-f39a-4239-8657-1c51d9dfdf6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:49.279090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fae1b8-f39a-4239-8657-1c51d9dfdf6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-c4fae1b8-f39a-4239-8657-1c51d9dfdf6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:49.284495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:49.285437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:49.286235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:49.286235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:49.286235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:49.286235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:49.286235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:49.286235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:49.286235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:49.286235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:49.286235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:49.286235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:49.286235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:36:49.286235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:49.286235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 214/444] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:36:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:49.287236 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fae1b8-f39a-4239-8657-1c51d9dfdf6d None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:49.305232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a9df6-c757-43f7-a07f-3f7099060e3c None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:49.332293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:49.346899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8878fa-dcc0-4ccd-a43c-a04eb71cda75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:49.347522 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8878fa-dcc0-4ccd-a43c-a04eb71cda75 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:49.347799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8878fa-dcc0-4ccd-a43c-a04eb71cda75 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:49.348066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8878fa-dcc0-4ccd-a43c-a04eb71cda75 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:49.348906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8878fa-dcc0-4ccd-a43c-a04eb71cda75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:49.349217 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8878fa-dcc0-4ccd-a43c-a04eb71cda75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:49.350774 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8878fa-dcc0-4ccd-a43c-a04eb71cda75 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:49.360358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 231/445] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:36:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:49.361215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8878fa-dcc0-4ccd-a43c-a04eb71cda75 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:49.406484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:49.410927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fae1b8-f39a-4239-8657-1c51d9dfdf6d None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:49.448729 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2728f-346b-405f-a85a-86843f40fec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:49.449344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2728f-346b-405f-a85a-86843f40fec1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:49.449631 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2728f-346b-405f-a85a-86843f40fec1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:49.449896 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2728f-346b-405f-a85a-86843f40fec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:49.450711 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2728f-346b-405f-a85a-86843f40fec1 None admin] RBAC: Adding query filter params (group.id=f5504a89d5b343d7bf5dac61e3410018, role.id=fe6c7625134a449c8395ab84ffd82371, scope.project.id=5c33c6fa415942299549b8fdce48c1b7) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:49.451019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2728f-346b-405f-a85a-86843f40fec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:49.452468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2728f-346b-405f-a85a-86843f40fec1 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:49.458488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 215/446] 10.222.0.86 () {56 vars in 1376 bytes} [Fri Aug 5 19:36:49 2022] GET /identity/v3/role_assignments?group.id=f5504a89d5b343d7bf5dac61e3410018&scope.project.id=5c33c6fa415942299549b8fdce48c1b7&role.id=fe6c7625134a449c8395ab84ffd82371 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:49.459318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2728f-346b-405f-a85a-86843f40fec1 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:49.487049 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8878fa-dcc0-4ccd-a43c-a04eb71cda75 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:49.543347 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc644c74-6a02-4fc5-88b4-f256cea4b59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:49.543987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc644c74-6a02-4fc5-88b4-f256cea4b59a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:49.544267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc644c74-6a02-4fc5-88b4-f256cea4b59a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:49.544540 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc644c74-6a02-4fc5-88b4-f256cea4b59a None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:49.545289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc644c74-6a02-4fc5-88b4-f256cea4b59a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:49.545593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc644c74-6a02-4fc5-88b4-f256cea4b59a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:49.547066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc644c74-6a02-4fc5-88b4-f256cea4b59a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:49.557191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 232/447] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:49.563377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc644c74-6a02-4fc5-88b4-f256cea4b59a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:49.610202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2728f-346b-405f-a85a-86843f40fec1 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:49.632765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:49.656879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0ccce-b1e9-49f6-ab3a-b32104815290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:49.656879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0ccce-b1e9-49f6-ab3a-b32104815290 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:49.657145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0ccce-b1e9-49f6-ab3a-b32104815290 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:49.657437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0ccce-b1e9-49f6-ab3a-b32104815290 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:49.696778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:49.707723 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc644c74-6a02-4fc5-88b4-f256cea4b59a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:49.751466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0ccce-b1e9-49f6-ab3a-b32104815290 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:49.752949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0ccce-b1e9-49f6-ab3a-b32104815290 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:49.756775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac3cff-7b9f-466b-a86d-bcd31d77fdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:49.757613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac3cff-7b9f-466b-a86d-bcd31d77fdbb None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:49.757896 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac3cff-7b9f-466b-a86d-bcd31d77fdbb None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:49.758160 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac3cff-7b9f-466b-a86d-bcd31d77fdbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:49.861348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac3cff-7b9f-466b-a86d-bcd31d77fdbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:49.863663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac3cff-7b9f-466b-a86d-bcd31d77fdbb None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:49.881780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 216/448] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:49.888631 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0ccce-b1e9-49f6-ab3a-b32104815290 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:49.923443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:49.956163 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:49.987390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 233/449] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:49.993480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac3cff-7b9f-466b-a86d-bcd31d77fdbb None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:50.003430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0ccce-b1e9-49f6-ab3a-b32104815290 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:50.042455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afa9e9-2dbd-4312-9a57-cc8f8ac0998b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:50.043125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afa9e9-2dbd-4312-9a57-cc8f8ac0998b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:50.043404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afa9e9-2dbd-4312-9a57-cc8f8ac0998b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:50.043683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afa9e9-2dbd-4312-9a57-cc8f8ac0998b None admin] PATH_INFO: `/v3/projects/5c33c6fa415942299549b8fdce48c1b7` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:50.044889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afa9e9-2dbd-4312-9a57-cc8f8ac0998b None admin] RBAC: Authorizing `identity:get_project(project_id=5c33c6fa415942299549b8fdce48c1b7)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:50.046340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afa9e9-2dbd-4312-9a57-cc8f8ac0998b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:50.049824 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 217/450] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:49 2022] GET /identity/v3/projects/5c33c6fa415942299549b8fdce48c1b7 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:50.053561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afa9e9-2dbd-4312-9a57-cc8f8ac0998b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:50.100623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac3cff-7b9f-466b-a86d-bcd31d77fdbb None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:50.138930 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e2583-4098-4816-bbe2-b7b0e2d6316b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:50.139597 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e2583-4098-4816-bbe2-b7b0e2d6316b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:50.139879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e2583-4098-4816-bbe2-b7b0e2d6316b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:50.140149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e2583-4098-4816-bbe2-b7b0e2d6316b None admin] PATH_INFO: `/v3/projects/11aa9c879322427d8920955a2aa93539` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:50.141322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e2583-4098-4816-bbe2-b7b0e2d6316b None admin] RBAC: Authorizing `identity:get_project(project_id=11aa9c879322427d8920955a2aa93539)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:50.142806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e2583-4098-4816-bbe2-b7b0e2d6316b None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:50.144481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 234/451] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:36:49 2022] GET /identity/v3/projects/11aa9c879322427d8920955a2aa93539 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:50.150240 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e2583-4098-4816-bbe2-b7b0e2d6316b None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:50.161768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afa9e9-2dbd-4312-9a57-cc8f8ac0998b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:50.169958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:50.199309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:50.205681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c61705-d17e-476e-8bd9-ffc45db28939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:50.206437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c61705-d17e-476e-8bd9-ffc45db28939 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:50.206744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c61705-d17e-476e-8bd9-ffc45db28939 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:50.207024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c61705-d17e-476e-8bd9-ffc45db28939 None admin] PATH_INFO: `/v3/groups/f5504a89d5b343d7bf5dac61e3410018` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:50.208200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c61705-d17e-476e-8bd9-ffc45db28939 None admin] RBAC: Authorizing `identity:get_group(group_id=f5504a89d5b343d7bf5dac61e3410018)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:50.209639 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c61705-d17e-476e-8bd9-ffc45db28939 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:50.211442 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 218/452] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:50 2022] GET /identity/v3/groups/f5504a89d5b343d7bf5dac61e3410018 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:50.217405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c61705-d17e-476e-8bd9-ffc45db28939 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:50.263858 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e2583-4098-4816-bbe2-b7b0e2d6316b None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:50.300897 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6006b-e5df-4757-b293-b6d6c6ebc839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:50.301542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6006b-e5df-4757-b293-b6d6c6ebc839 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:50.301824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6006b-e5df-4757-b293-b6d6c6ebc839 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:50.302100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6006b-e5df-4757-b293-b6d6c6ebc839 None admin] PATH_INFO: `/v3/groups/2b5cd417939b46cda3fee3d3bba700e9` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:50.303336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6006b-e5df-4757-b293-b6d6c6ebc839 None admin] RBAC: Authorizing `identity:get_group(group_id=2b5cd417939b46cda3fee3d3bba700e9)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:50.305368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6006b-e5df-4757-b293-b6d6c6ebc839 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:50.307092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 235/453] 10.222.0.86 () {56 vars in 1180 bytes} [Fri Aug 5 19:36:50 2022] GET /identity/v3/groups/2b5cd417939b46cda3fee3d3bba700e9 => generated 229 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:50.313368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6006b-e5df-4757-b293-b6d6c6ebc839 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:50.325014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c61705-d17e-476e-8bd9-ffc45db28939 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:50.361982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4bc8d-a824-4e16-a2ec-d1c57f47e358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:50.362618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4bc8d-a824-4e16-a2ec-d1c57f47e358 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:50.362922 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4bc8d-a824-4e16-a2ec-d1c57f47e358 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:50.363195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4bc8d-a824-4e16-a2ec-d1c57f47e358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:50.363981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4bc8d-a824-4e16-a2ec-d1c57f47e358 None admin] RBAC: Adding query filter params (group.id=f5504a89d5b343d7bf5dac61e3410018, role.id=686764f0686e446f884315081e158bf3, scope.project.id=5c33c6fa415942299549b8fdce48c1b7) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:50.364290 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4bc8d-a824-4e16-a2ec-d1c57f47e358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:50.365724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4bc8d-a824-4e16-a2ec-d1c57f47e358 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:50.373803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 219/454] 10.222.0.86 () {56 vars in 1376 bytes} [Fri Aug 5 19:36:50 2022] GET /identity/v3/role_assignments?group.id=f5504a89d5b343d7bf5dac61e3410018&scope.project.id=5c33c6fa415942299549b8fdce48c1b7&role.id=686764f0686e446f884315081e158bf3 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:50.411637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:50.432180 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6006b-e5df-4757-b293-b6d6c6ebc839 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:50.472606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd859d-9089-4de6-b33f-1509fc41ccf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:50.473208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd859d-9089-4de6-b33f-1509fc41ccf0 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:50.473487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd859d-9089-4de6-b33f-1509fc41ccf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:50.473756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd859d-9089-4de6-b33f-1509fc41ccf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:50.474566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd859d-9089-4de6-b33f-1509fc41ccf0 None admin] RBAC: Adding query filter params (group.id=2b5cd417939b46cda3fee3d3bba700e9, role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=11aa9c879322427d8920955a2aa93539) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:50.475362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd859d-9089-4de6-b33f-1509fc41ccf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:50.478053 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd859d-9089-4de6-b33f-1509fc41ccf0 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:50.485316 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 236/455] 10.222.0.86 () {56 vars in 1376 bytes} [Fri Aug 5 19:36:50 2022] GET /identity/v3/role_assignments?group.id=2b5cd417939b46cda3fee3d3bba700e9&scope.project.id=11aa9c879322427d8920955a2aa93539&role.id=93eb204498fc49d297bcd1841e00818d => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:53.242097 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485907f3-9806-4cbd-b1d4-5c4682937fe0 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:53.242618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485907f3-9806-4cbd-b1d4-5c4682937fe0 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:53.242715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485907f3-9806-4cbd-b1d4-5c4682937fe0 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:53.244254 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 220/456] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:53.249760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b4fd3-d900-4fb1-84a8-2fdfcbcb56b5 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:53.250612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b4fd3-d900-4fb1-84a8-2fdfcbcb56b5 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:53.251119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b4fd3-d900-4fb1-84a8-2fdfcbcb56b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:53.279798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-547b4fd3-d900-4fb1-84a8-2fdfcbcb56b5 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:53.281382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-547b4fd3-d900-4fb1-84a8-2fdfcbcb56b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:53.368320 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 237/457] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:53.417607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485907f3-9806-4cbd-b1d4-5c4682937fe0 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:53.521489 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5926fa-b16f-4f5e-a3cd-283dade093ac None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:53.521804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:53.523280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5926fa-b16f-4f5e-a3cd-283dade093ac None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:53.523795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5926fa-b16f-4f5e-a3cd-283dade093ac None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:53.525799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 238/458] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:53.531656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c0504-286d-4dfd-8945-36c864f22f4e None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:53.532254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c0504-286d-4dfd-8945-36c864f22f4e None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:53.532746 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c0504-286d-4dfd-8945-36c864f22f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:53.563993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-ca2c0504-286d-4dfd-8945-36c864f22f4e None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:53.567164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ca2c0504-286d-4dfd-8945-36c864f22f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:53.604189 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485907f3-9806-4cbd-b1d4-5c4682937fe0 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:53.644692 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14d651-ae85-4cf0-98d6-b037d66d5a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:53.646254 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14d651-ae85-4cf0-98d6-b037d66d5a86 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:53.646901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14d651-ae85-4cf0-98d6-b037d66d5a86 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:53.647376 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14d651-ae85-4cf0-98d6-b037d66d5a86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:53.665274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14d651-ae85-4cf0-98d6-b037d66d5a86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:53.666798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14d651-ae85-4cf0-98d6-b037d66d5a86 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:53.679208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 239/459] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:53.680270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22383988-50e9-49ec-8b89-12ec84404746 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:53.680560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22383988-50e9-49ec-8b89-12ec84404746 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:53.680838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22383988-50e9-49ec-8b89-12ec84404746 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:53.682391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 240/460] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-4b14d651-ae85-4cf0-98d6-b037d66d5a86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:36:53.691400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:53.692310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:53.693184 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:36:53.693184 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:36:53.693184 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:53.693184 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:53.693184 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 221/461] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:53.699642 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04661171-b2c2-48d7-8fac-34621ef0dfe0 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:53.699929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04661171-b2c2-48d7-8fac-34621ef0dfe0 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:53.700215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04661171-b2c2-48d7-8fac-34621ef0dfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:53.704580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14d651-ae85-4cf0-98d6-b037d66d5a86 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:53.732473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-04661171-b2c2-48d7-8fac-34621ef0dfe0 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:53.733732 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-04661171-b2c2-48d7-8fac-34621ef0dfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:53.736274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:53.830819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 241/462] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:53.831983 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04661171-b2c2-48d7-8fac-34621ef0dfe0 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:53.872895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14d651-ae85-4cf0-98d6-b037d66d5a86 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:53.897744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:53.926310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f4195-6e9b-4e6b-b4c4-ba8dd8939857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:53.927305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f4195-6e9b-4e6b-b4c4-ba8dd8939857 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:53.927766 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f4195-6e9b-4e6b-b4c4-ba8dd8939857 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:53.928237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f4195-6e9b-4e6b-b4c4-ba8dd8939857 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:53.930408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f4195-6e9b-4e6b-b4c4-ba8dd8939857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:53.931228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f4195-6e9b-4e6b-b4c4-ba8dd8939857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:53.933863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f4195-6e9b-4e6b-b4c4-ba8dd8939857 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:53.951688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 222/463] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:53.957468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f4195-6e9b-4e6b-b4c4-ba8dd8939857 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:53.971532 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04661171-b2c2-48d7-8fac-34621ef0dfe0 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:54.014440 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c2ff9-45c2-4b3e-8192-02c27d8cdaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:54.015123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2ff9-45c2-4b3e-8192-02c27d8cdaa2 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:54.015394 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2ff9-45c2-4b3e-8192-02c27d8cdaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:54.015659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c2ff9-45c2-4b3e-8192-02c27d8cdaa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:54.023975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:54.028524 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c2ff9-45c2-4b3e-8192-02c27d8cdaa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:54.030666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c2ff9-45c2-4b3e-8192-02c27d8cdaa2 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-9c8c2ff9-45c2-4b3e-8192-02c27d8cdaa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:36:54.049089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:54.050069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:54.051067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:36:54.051067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:36:54.051067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:54.051067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:54.051067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 242/464] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:54.051327 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c2ff9-45c2-4b3e-8192-02c27d8cdaa2 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:54.086378 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f4195-6e9b-4e6b-b4c4-ba8dd8939857 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:54.127888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a3147-72f6-4482-b1f0-d946b16fc4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:54.128512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a3147-72f6-4482-b1f0-d946b16fc4ca None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:54.128787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a3147-72f6-4482-b1f0-d946b16fc4ca None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:54.129056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a3147-72f6-4482-b1f0-d946b16fc4ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:54.147092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a3147-72f6-4482-b1f0-d946b16fc4ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:54.149729 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a3147-72f6-4482-b1f0-d946b16fc4ca None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:54.174079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c2ff9-45c2-4b3e-8192-02c27d8cdaa2 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-784a3147-72f6-4482-b1f0-d946b16fc4ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:36:54.181165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:54.182083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:54.188484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:36:54.188484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:36:54.188484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:54.188484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:54.188484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 223/465] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:54.188484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d630-0fe5-4f79-8dd7-dfe31c419952 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:54.188484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d630-0fe5-4f79-8dd7-dfe31c419952 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:54.188484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d630-0fe5-4f79-8dd7-dfe31c419952 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:54.188484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 224/466] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:54.189165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236d630-0fe5-4f79-8dd7-dfe31c419952 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:54.224994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85aa71e-fb4d-41a0-b320-deb5d9a5d2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:54.226183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85aa71e-fb4d-41a0-b320-deb5d9a5d2d7 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:54.226840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85aa71e-fb4d-41a0-b320-deb5d9a5d2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:54.227129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85aa71e-fb4d-41a0-b320-deb5d9a5d2d7 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:54.228176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:54.229023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85aa71e-fb4d-41a0-b320-deb5d9a5d2d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:54.231711 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85aa71e-fb4d-41a0-b320-deb5d9a5d2d7 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:54.274685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 243/467] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:36:54 2022] POST /identity/v3/users => generated 246 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:36:54.274685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85aa71e-fb4d-41a0-b320-deb5d9a5d2d7 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:54.323543 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236d630-0fe5-4f79-8dd7-dfe31c419952 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:54.363292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:54.386612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924193f-7b3b-4739-96ca-8793fff06f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:54.388351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924193f-7b3b-4739-96ca-8793fff06f40 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:54.388844 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924193f-7b3b-4739-96ca-8793fff06f40 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:54.389259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924193f-7b3b-4739-96ca-8793fff06f40 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:54.390243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924193f-7b3b-4739-96ca-8793fff06f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:54.390712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924193f-7b3b-4739-96ca-8793fff06f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:54.393146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924193f-7b3b-4739-96ca-8793fff06f40 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:54.420883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 225/468] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:54.420883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24478b-ff4c-400d-b2c4-8f8bbc29bd7b None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:54.420883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24478b-ff4c-400d-b2c4-8f8bbc29bd7b None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:54.420883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24478b-ff4c-400d-b2c4-8f8bbc29bd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:54.448257 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-bf24478b-ff4c-400d-b2c4-8f8bbc29bd7b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:54.449799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-bf24478b-ff4c-400d-b2c4-8f8bbc29bd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:54.450534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85aa71e-fb4d-41a0-b320-deb5d9a5d2d7 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:54.505783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e1a5e-392d-4498-8b0e-ad0245ea9856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:54.506434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e1a5e-392d-4498-8b0e-ad0245ea9856 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:54.508224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e1a5e-392d-4498-8b0e-ad0245ea9856 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:54.508224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e1a5e-392d-4498-8b0e-ad0245ea9856 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:54.508224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e1a5e-392d-4498-8b0e-ad0245ea9856 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:54.508224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e1a5e-392d-4498-8b0e-ad0245ea9856 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:54.509460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e1a5e-392d-4498-8b0e-ad0245ea9856 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:54.525009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 244/469] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:54.525679 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e1a5e-392d-4498-8b0e-ad0245ea9856 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:54.549073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 226/470] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:54.551563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24478b-ff4c-400d-b2c4-8f8bbc29bd7b None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:54.573450 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:54.655087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e1a5e-392d-4498-8b0e-ad0245ea9856 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:54.663906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:54.672539 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24478b-ff4c-400d-b2c4-8f8bbc29bd7b None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:54.701763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5ce7f-6c02-4e03-be98-72e5a716d058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:54.702383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5ce7f-6c02-4e03-be98-72e5a716d058 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:54.702687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5ce7f-6c02-4e03-be98-72e5a716d058 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:54.702968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5ce7f-6c02-4e03-be98-72e5a716d058 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:54.703650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5ce7f-6c02-4e03-be98-72e5a716d058 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:54.705367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5ce7f-6c02-4e03-be98-72e5a716d058 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:54.719309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1bc0cb-156e-4e22-9a70-b5055fd2cebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:54.720212 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bc0cb-156e-4e22-9a70-b5055fd2cebf None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:54.720834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bc0cb-156e-4e22-9a70-b5055fd2cebf None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:54.721271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bc0cb-156e-4e22-9a70-b5055fd2cebf None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:54.722673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1bc0cb-156e-4e22-9a70-b5055fd2cebf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:54.725458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1bc0cb-156e-4e22-9a70-b5055fd2cebf None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:54.743174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 245/471] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:36:54 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:36:54.744940 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503240ae-ae62-4bf3-b35d-b142de893fe5 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:54.745217 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503240ae-ae62-4bf3-b35d-b142de893fe5 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:54.745563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503240ae-ae62-4bf3-b35d-b142de893fe5 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:54.749777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 246/472] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:54.751387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503240ae-ae62-4bf3-b35d-b142de893fe5 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:54.758597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 227/473] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:36:54 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:36:54.760431 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96520c1a-4fac-4d9a-9479-1bf7b556511e None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:54.760934 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96520c1a-4fac-4d9a-9479-1bf7b556511e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:54.761400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96520c1a-4fac-4d9a-9479-1bf7b556511e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:54.782880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-96520c1a-4fac-4d9a-9479-1bf7b556511e None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:54.784665 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-96520c1a-4fac-4d9a-9479-1bf7b556511e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:54.898492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 228/474] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:54.908749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503240ae-ae62-4bf3-b35d-b142de893fe5 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:54.947220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96520c1a-4fac-4d9a-9479-1bf7b556511e None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:54.951001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:54.971429 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16370402-34e0-4b47-99da-6077cc572b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:54.972086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16370402-34e0-4b47-99da-6077cc572b5c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:54.972364 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16370402-34e0-4b47-99da-6077cc572b5c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:54.972628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16370402-34e0-4b47-99da-6077cc572b5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:54.993773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16370402-34e0-4b47-99da-6077cc572b5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:54.995390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16370402-34e0-4b47-99da-6077cc572b5c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-16370402-34e0-4b47-99da-6077cc572b5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:36:55.015751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:55.016667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:55.017509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:36:55.017509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:36:55.017509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:55.017509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:55.017509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 247/475] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:55.023575 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16370402-34e0-4b47-99da-6077cc572b5c None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:55.046549 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:55.102542 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96520c1a-4fac-4d9a-9479-1bf7b556511e None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:55.161194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b68739-2e7d-4c2c-89b9-a018bffebcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:55.161901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b68739-2e7d-4c2c-89b9-a018bffebcf9 None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:55.165748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b68739-2e7d-4c2c-89b9-a018bffebcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:55.165748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b68739-2e7d-4c2c-89b9-a018bffebcf9 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:55.165748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b68739-2e7d-4c2c-89b9-a018bffebcf9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:55.165748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b68739-2e7d-4c2c-89b9-a018bffebcf9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:55.176118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.sql.core [None req-79b68739-2e7d-4c2c-89b9-a018bffebcf9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 19:36:55.176118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 19:36:55.176118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [parameters: {'id': '717a4d90ab034581810b54e811889b16', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 19:36:55.176118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87005) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 19:36:55.184863 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16370402-34e0-4b47-99da-6077cc572b5c None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-79b68739-2e7d-4c2c-89b9-a018bffebcf9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application result.read() Aug 05 19:36:55.196047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return ref.to_dict() Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:36:55.197194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application yield resource Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise self.value Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application session.commit() Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 19:36:55.198418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self.session.flush() Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application compat.raise_( Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 19:36:55.199715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:36:55.200810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application result.read() Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:36:55.201823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [parameters: {'id': '717a4d90ab034581810b54e811889b16', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:55.204008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 19:36:55.204959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:55.206053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 229/476] 10.222.0.86 () {60 vars in 1131 bytes} [Fri Aug 5 19:36:54 2022] POST /identity/v3/roles => generated 144 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:55.207260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b68739-2e7d-4c2c-89b9-a018bffebcf9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:55.253124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb464141-214a-4b2c-b367-64a078e93d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:55.253947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb464141-214a-4b2c-b367-64a078e93d4a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:55.254352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb464141-214a-4b2c-b367-64a078e93d4a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:55.254762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb464141-214a-4b2c-b367-64a078e93d4a None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:55.255627 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb464141-214a-4b2c-b367-64a078e93d4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:55.256031 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb464141-214a-4b2c-b367-64a078e93d4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:55.259256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb464141-214a-4b2c-b367-64a078e93d4a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:55.273705 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 248/477] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:55.293745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb464141-214a-4b2c-b367-64a078e93d4a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:55.305736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:55.345697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b68739-2e7d-4c2c-89b9-a018bffebcf9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:55.388020 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b643d-a4ed-47fe-955e-5ecd2f3b47c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:55.388683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b643d-a4ed-47fe-955e-5ecd2f3b47c1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:55.388966 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b643d-a4ed-47fe-955e-5ecd2f3b47c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:55.389242 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b643d-a4ed-47fe-955e-5ecd2f3b47c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:55.403471 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b643d-a4ed-47fe-955e-5ecd2f3b47c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-d31b643d-a4ed-47fe-955e-5ecd2f3b47c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:55.408833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:55.410527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:55.411530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:55.411530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:55.411530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:55.411530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:55.411530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:55.411530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:55.411530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:55.411530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:55.411530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:55.411530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:55.411530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:36:55.411530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:55.411530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 230/478] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:36:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:55.417572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b643d-a4ed-47fe-955e-5ecd2f3b47c1 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:55.537756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b643d-a4ed-47fe-955e-5ecd2f3b47c1 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:55.560352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:55.581753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5615a3f-a351-4270-b721-0aa5ead148b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:55.582380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5615a3f-a351-4270-b721-0aa5ead148b9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:55.582960 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5615a3f-a351-4270-b721-0aa5ead148b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:55.583248 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5615a3f-a351-4270-b721-0aa5ead148b9 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:55.583980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5615a3f-a351-4270-b721-0aa5ead148b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:55.584289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5615a3f-a351-4270-b721-0aa5ead148b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:55.586511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5615a3f-a351-4270-b721-0aa5ead148b9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:55.597028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 231/479] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:36:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:36:55.597519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:55.643168 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb464141-214a-4b2c-b367-64a078e93d4a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:55.706025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e149373-8fb7-4745-9d24-1fd455c32363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:55.706577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e149373-8fb7-4745-9d24-1fd455c32363 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:55.706881 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e149373-8fb7-4745-9d24-1fd455c32363 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:55.707154 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e149373-8fb7-4745-9d24-1fd455c32363 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:55.707849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e149373-8fb7-4745-9d24-1fd455c32363 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:55.709315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e149373-8fb7-4745-9d24-1fd455c32363 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:55.736480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 249/480] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:36:55 2022] POST /identity/v3/users => generated 249 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:36:56.828095 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9d154-28fe-495a-a6db-dcdd191beda5 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:56.829295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9d154-28fe-495a-a6db-dcdd191beda5 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:56.829777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9d154-28fe-495a-a6db-dcdd191beda5 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:56.831855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 232/481] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:56.837784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4c518-6f5c-43ca-906a-53ca727344e8 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:56.838710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4c518-6f5c-43ca-906a-53ca727344e8 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:56.839293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4c518-6f5c-43ca-906a-53ca727344e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:56.870379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-2fa4c518-6f5c-43ca-906a-53ca727344e8 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:56.872094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2fa4c518-6f5c-43ca-906a-53ca727344e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:56.961637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 250/482] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:57.009197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9d154-28fe-495a-a6db-dcdd191beda5 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:57.161964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9d154-28fe-495a-a6db-dcdd191beda5 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:57.228762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bf978-8295-4e37-bc89-134013f92777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:57.229410 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bf978-8295-4e37-bc89-134013f92777 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:57.229745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bf978-8295-4e37-bc89-134013f92777 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:57.230019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bf978-8295-4e37-bc89-134013f92777 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:57.255347 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bf978-8295-4e37-bc89-134013f92777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-1b6bf978-8295-4e37-bc89-134013f92777 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:57.262831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:57.263828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:57.264783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:57.264783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:57.264783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:57.264783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:57.264783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:57.264783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:57.264783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:57.264783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:57.264783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:57.264783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:57.264783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:36:57.264783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:57.265405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 233/483] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:36:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:57.270746 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4c518-6f5c-43ca-906a-53ca727344e8 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:57.375121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:57.417743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284133-9ef2-40b7-b37f-611a4d2d4c9e None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:57.417920 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284133-9ef2-40b7-b37f-611a4d2d4c9e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:57.418197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284133-9ef2-40b7-b37f-611a4d2d4c9e None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:57.421213 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 234/484] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:57.425618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ffc60-2300-4f09-b268-0ff76b2f91a6 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:57.425901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ffc60-2300-4f09-b268-0ff76b2f91a6 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:57.426167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ffc60-2300-4f09-b268-0ff76b2f91a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:57.431922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa4c518-6f5c-43ca-906a-53ca727344e8 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:57.474376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29197cf1-799b-4f37-8096-4e9d3d4003c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:57.475012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-2d8ffc60-2300-4f09-b268-0ff76b2f91a6 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:57.475126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29197cf1-799b-4f37-8096-4e9d3d4003c7 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:57.475307 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29197cf1-799b-4f37-8096-4e9d3d4003c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:57.475626 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29197cf1-799b-4f37-8096-4e9d3d4003c7 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:57.479090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2d8ffc60-2300-4f09-b268-0ff76b2f91a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:57.479245 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29197cf1-799b-4f37-8096-4e9d3d4003c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:57.479245 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29197cf1-799b-4f37-8096-4e9d3d4003c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:57.479245 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29197cf1-799b-4f37-8096-4e9d3d4003c7 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:57.488659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 251/485] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:36:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:57.494131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29197cf1-799b-4f37-8096-4e9d3d4003c7 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:57.624310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:57.630671 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29197cf1-799b-4f37-8096-4e9d3d4003c7 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:57.668231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782f648-03a3-4e02-a019-3523032a917d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:57.668795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782f648-03a3-4e02-a019-3523032a917d None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:57.669075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782f648-03a3-4e02-a019-3523032a917d None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:57.669346 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782f648-03a3-4e02-a019-3523032a917d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:57.689375 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 235/486] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:36:57.691476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782f648-03a3-4e02-a019-3523032a917d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:57.691977 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:57.702967 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782f648-03a3-4e02-a019-3523032a917d None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:57.713097 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ffc60-2300-4f09-b268-0ff76b2f91a6 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-a782f648-03a3-4e02-a019-3523032a917d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:36:57.723891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:57.724810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:57.731823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:36:57.731823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:36:57.731823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:57.731823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:57.731823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 252/487] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:36:57.739697 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782f648-03a3-4e02-a019-3523032a917d None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:57.841902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ffc60-2300-4f09-b268-0ff76b2f91a6 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:57.896160 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782f648-03a3-4e02-a019-3523032a917d None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:57.896720 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eeb9f6-91c0-43c0-98b8-7621b4094b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:57.897221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eeb9f6-91c0-43c0-98b8-7621b4094b0c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:57.898125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eeb9f6-91c0-43c0-98b8-7621b4094b0c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:57.898535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eeb9f6-91c0-43c0-98b8-7621b4094b0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:57.915383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eeb9f6-91c0-43c0-98b8-7621b4094b0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-e2eeb9f6-91c0-43c0-98b8-7621b4094b0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:57.921963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:57.922894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 236/488] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:36:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:57.927334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24370408-8b20-4842-b65d-e54ff726bc44 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:57.929908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24370408-8b20-4842-b65d-e54ff726bc44 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:57.929908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24370408-8b20-4842-b65d-e54ff726bc44 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:57.931534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 237/489] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:57.934018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff80a0-bb58-4f1b-92f3-66b8a3d18083 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:57.934582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff80a0-bb58-4f1b-92f3-66b8a3d18083 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:57.934877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff80a0-bb58-4f1b-92f3-66b8a3d18083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:57.949824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb75e6b-bc2e-4d53-82e4-41ff46414d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:57.950833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb75e6b-bc2e-4d53-82e4-41ff46414d69 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:57.951287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb75e6b-bc2e-4d53-82e4-41ff46414d69 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:57.951763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb75e6b-bc2e-4d53-82e4-41ff46414d69 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:57.952700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb75e6b-bc2e-4d53-82e4-41ff46414d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:57.953209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb75e6b-bc2e-4d53-82e4-41ff46414d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:57.954908 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb75e6b-bc2e-4d53-82e4-41ff46414d69 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:57.958537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-72ff80a0-bb58-4f1b-92f3-66b8a3d18083 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:57.959778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-72ff80a0-bb58-4f1b-92f3-66b8a3d18083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:57.975065 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 253/490] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:57.976387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb75e6b-bc2e-4d53-82e4-41ff46414d69 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:58.001846 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:58.023570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:58.049143 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 238/491] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:58.050313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff80a0-bb58-4f1b-92f3-66b8a3d18083 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:58.136541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb75e6b-bc2e-4d53-82e4-41ff46414d69 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:58.175433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f374b1-60b8-478e-82b2-e8f2c9fa14f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:58.175907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f374b1-60b8-478e-82b2-e8f2c9fa14f3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:58.176184 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f374b1-60b8-478e-82b2-e8f2c9fa14f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:58.176452 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f374b1-60b8-478e-82b2-e8f2c9fa14f3 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:58.177178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f374b1-60b8-478e-82b2-e8f2c9fa14f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:58.177480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f374b1-60b8-478e-82b2-e8f2c9fa14f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:58.179567 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f374b1-60b8-478e-82b2-e8f2c9fa14f3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:58.194809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 254/492] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:36:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:58.194809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f374b1-60b8-478e-82b2-e8f2c9fa14f3 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:58.206095 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff80a0-bb58-4f1b-92f3-66b8a3d18083 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:58.246851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e284d5f-5d6b-4849-9872-827553e14df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:58.248014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e284d5f-5d6b-4849-9872-827553e14df2 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:58.248525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e284d5f-5d6b-4849-9872-827553e14df2 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:58.248996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e284d5f-5d6b-4849-9872-827553e14df2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:58.258313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e284d5f-5d6b-4849-9872-827553e14df2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:58.260323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e284d5f-5d6b-4849-9872-827553e14df2 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-3e284d5f-5d6b-4849-9872-827553e14df2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:36:58.275258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:58.276301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:58.277188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:58.277188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:58.277188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:36:58.277188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:36:58.277188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:36:58.277188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:36:58.277188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:36:58.277188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:36:58.277188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:36:58.277188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:58.284346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 239/493] 10.222.0.86 () {56 vars in 1093 bytes} [Fri Aug 5 19:36:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:58.285685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e284d5f-5d6b-4849-9872-827553e14df2 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:58.300109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:58.306688 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f374b1-60b8-478e-82b2-e8f2c9fa14f3 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:58.323508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:58.346058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105751e-75b1-4203-8816-5b7f8e290e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:58.347003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105751e-75b1-4203-8816-5b7f8e290e81 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:58.347289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105751e-75b1-4203-8816-5b7f8e290e81 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:58.347653 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105751e-75b1-4203-8816-5b7f8e290e81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:58.360107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105751e-75b1-4203-8816-5b7f8e290e81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-3105751e-75b1-4203-8816-5b7f8e290e81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:58.365457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:58.366952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:58.367790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:58.367790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:58.367790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:58.367790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:58.367790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:58.367790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:58.367790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:58.367790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:58.367790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:58.367790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:58.367790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:36:58.367790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:58.372435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 255/494] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:36:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:58.373460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105751e-75b1-4203-8816-5b7f8e290e81 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:58.417117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e284d5f-5d6b-4849-9872-827553e14df2 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:58.456635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af756f6b-2157-4260-9826-2448f1d3b376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:58.457198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af756f6b-2157-4260-9826-2448f1d3b376 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:58.457475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af756f6b-2157-4260-9826-2448f1d3b376 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:58.457745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af756f6b-2157-4260-9826-2448f1d3b376 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:58.472136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af756f6b-2157-4260-9826-2448f1d3b376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:58.473618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af756f6b-2157-4260-9826-2448f1d3b376 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-af756f6b-2157-4260-9826-2448f1d3b376 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:36:58.490436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:58.491436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:58.492437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:36:58.492437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:36:58.492437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:58.492437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:58.492437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 240/495] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:58.495469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af756f6b-2157-4260-9826-2448f1d3b376 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:58.504011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105751e-75b1-4203-8816-5b7f8e290e81 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:58.542230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53b8f8-5a20-4da3-9433-5900b575d7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:58.542879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53b8f8-5a20-4da3-9433-5900b575d7d7 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:58.543158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53b8f8-5a20-4da3-9433-5900b575d7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:58.543419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53b8f8-5a20-4da3-9433-5900b575d7d7 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:58.544225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53b8f8-5a20-4da3-9433-5900b575d7d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:58.544524 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53b8f8-5a20-4da3-9433-5900b575d7d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:58.545981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53b8f8-5a20-4da3-9433-5900b575d7d7 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:58.584727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 256/496] 10.222.0.86 () {56 vars in 1133 bytes} [Fri Aug 5 19:36:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:58.586191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d9ee3-e957-48c7-8020-46c72eb86580 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:58.586472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d9ee3-e957-48c7-8020-46c72eb86580 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:58.586775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d9ee3-e957-48c7-8020-46c72eb86580 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:58.588094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 257/497] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:58.588802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d9ee3-e957-48c7-8020-46c72eb86580 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:58.592187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:58.605526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:58.612708 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af756f6b-2157-4260-9826-2448f1d3b376 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:58.650079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c816127-fe75-4a43-8618-9dc25c48cd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:58.650749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c816127-fe75-4a43-8618-9dc25c48cd13 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:58.651112 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c816127-fe75-4a43-8618-9dc25c48cd13 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:58.651394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c816127-fe75-4a43-8618-9dc25c48cd13 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:58.652101 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c816127-fe75-4a43-8618-9dc25c48cd13 None admin] RBAC: Adding query filter params (name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:58.652398 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c816127-fe75-4a43-8618-9dc25c48cd13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:58.653807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c816127-fe75-4a43-8618-9dc25c48cd13 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:58.662730 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 241/498] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:36:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:58.664852 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad18514-eb32-475a-882f-81e0f69d5b6a None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:58.665134 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad18514-eb32-475a-882f-81e0f69d5b6a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:58.665406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad18514-eb32-475a-882f-81e0f69d5b6a None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:58.670572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 242/499] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:36:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:36:58.671385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad18514-eb32-475a-882f-81e0f69d5b6a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:58.697818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d9ee3-e957-48c7-8020-46c72eb86580 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:58.734213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa0ef0-8e5e-45fc-9484-bbac93a6ca23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:58.734758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa0ef0-8e5e-45fc-9484-bbac93a6ca23 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:58.735032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa0ef0-8e5e-45fc-9484-bbac93a6ca23 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:58.735300 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa0ef0-8e5e-45fc-9484-bbac93a6ca23 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:58.735987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa0ef0-8e5e-45fc-9484-bbac93a6ca23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:58.736287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa0ef0-8e5e-45fc-9484-bbac93a6ca23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:58.737654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa0ef0-8e5e-45fc-9484-bbac93a6ca23 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:58.751009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 258/500] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:58.752308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535b1d7-8967-4017-9578-e145d22c396a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:58.752592 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535b1d7-8967-4017-9578-e145d22c396a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:58.752859 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535b1d7-8967-4017-9578-e145d22c396a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:58.774284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-0535b1d7-8967-4017-9578-e145d22c396a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:58.775546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-0535b1d7-8967-4017-9578-e145d22c396a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:58.778974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad18514-eb32-475a-882f-81e0f69d5b6a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:58.815886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743557e-e580-4ea2-98a0-9496c621842a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:58.816502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743557e-e580-4ea2-98a0-9496c621842a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:58.816777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743557e-e580-4ea2-98a0-9496c621842a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:58.817042 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743557e-e580-4ea2-98a0-9496c621842a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:58.830847 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:58.849087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 259/501] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:58.849942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535b1d7-8967-4017-9578-e145d22c396a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:58.864559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:58.891978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743557e-e580-4ea2-98a0-9496c621842a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:58.893443 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743557e-e580-4ea2-98a0-9496c621842a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:58.969860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535b1d7-8967-4017-9578-e145d22c396a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:59.003109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 243/502] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:36:59.004396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c61121-4b01-4133-9c4c-7f0e81fcd556 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:59.004688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c61121-4b01-4133-9c4c-7f0e81fcd556 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:59.004963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c61121-4b01-4133-9c4c-7f0e81fcd556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:59.008234 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d505aaa-e8ad-4b8d-8c26-6847120122de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:59.008935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d505aaa-e8ad-4b8d-8c26-6847120122de None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:59.009218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d505aaa-e8ad-4b8d-8c26-6847120122de None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:59.009495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d505aaa-e8ad-4b8d-8c26-6847120122de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:59.021022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d505aaa-e8ad-4b8d-8c26-6847120122de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:59.023276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d505aaa-e8ad-4b8d-8c26-6847120122de None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:59.025233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-f7c61121-4b01-4133-9c4c-7f0e81fcd556 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:36:59.026454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f7c61121-4b01-4133-9c4c-7f0e81fcd556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-1d505aaa-e8ad-4b8d-8c26-6847120122de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:36:59.038608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:59.039640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:59.040580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:36:59.040580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:36:59.040580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:59.040580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:59.040580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 260/503] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:59.041229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d505aaa-e8ad-4b8d-8c26-6847120122de None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:59.067121 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:59.100980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 244/504] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:36:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:36:59.101827 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c61121-4b01-4133-9c4c-7f0e81fcd556 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:59.116938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:59.165939 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d505aaa-e8ad-4b8d-8c26-6847120122de None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:59.201617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341386b-4b41-4cfe-8d55-02231635aeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:59.202255 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341386b-4b41-4cfe-8d55-02231635aeb8 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:59.202558 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341386b-4b41-4cfe-8d55-02231635aeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:59.202967 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341386b-4b41-4cfe-8d55-02231635aeb8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:59.209085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c61121-4b01-4133-9c4c-7f0e81fcd556 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:59.210450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341386b-4b41-4cfe-8d55-02231635aeb8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:59.212320 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341386b-4b41-4cfe-8d55-02231635aeb8 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-8341386b-4b41-4cfe-8d55-02231635aeb8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:36:59.226185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:59.227356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:59.228606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:59.228606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:59.228606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:36:59.228606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:36:59.228606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:36:59.228606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:36:59.228606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:36:59.228606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:36:59.228606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:36:59.228606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:59.228606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 261/505] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:59.230261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341386b-4b41-4cfe-8d55-02231635aeb8 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:59.248976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e69d55-3f9c-4a4d-92ec-64d44fb588cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:59.249784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e69d55-3f9c-4a4d-92ec-64d44fb588cf None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:59.250185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e69d55-3f9c-4a4d-92ec-64d44fb588cf None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:59.250520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e69d55-3f9c-4a4d-92ec-64d44fb588cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:59.261896 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e69d55-3f9c-4a4d-92ec-64d44fb588cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:59.263396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e69d55-3f9c-4a4d-92ec-64d44fb588cf None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-d5e69d55-3f9c-4a4d-92ec-64d44fb588cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:36:59.278360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:59.284715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:59.285780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:36:59.285780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:36:59.285780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:59.285780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:36:59.285780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 245/506] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:59.285780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e69d55-3f9c-4a4d-92ec-64d44fb588cf None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:59.315578 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:59.347192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341386b-4b41-4cfe-8d55-02231635aeb8 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:59.364362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:59.385660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4334a-594d-4549-960b-3a0a94614aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:59.386294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4334a-594d-4549-960b-3a0a94614aed None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:59.386565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4334a-594d-4549-960b-3a0a94614aed None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:59.386866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4334a-594d-4549-960b-3a0a94614aed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:59.396982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e69d55-3f9c-4a4d-92ec-64d44fb588cf None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:59.400164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4334a-594d-4549-960b-3a0a94614aed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:59.402133 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4334a-594d-4549-960b-3a0a94614aed None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-fde4334a-594d-4549-960b-3a0a94614aed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:36:59.417805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:59.418862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:59.419965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:36:59.419965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:36:59.419965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:36:59.419965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:59.419965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 262/507] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:59.420434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4334a-594d-4549-960b-3a0a94614aed None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:59.439710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705acd1f-384a-463a-a22f-d9d25555cf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:59.440319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705acd1f-384a-463a-a22f-d9d25555cf58 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:59.440592 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705acd1f-384a-463a-a22f-d9d25555cf58 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:59.440866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705acd1f-384a-463a-a22f-d9d25555cf58 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:59.441542 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705acd1f-384a-463a-a22f-d9d25555cf58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:59.441841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705acd1f-384a-463a-a22f-d9d25555cf58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:59.443237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705acd1f-384a-463a-a22f-d9d25555cf58 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:59.455757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 246/508] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:59.456590 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705acd1f-384a-463a-a22f-d9d25555cf58 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:59.533325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4334a-594d-4549-960b-3a0a94614aed None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:59.549575 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:59.565621 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705acd1f-384a-463a-a22f-d9d25555cf58 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:59.568077 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4e395-ef92-4842-90d2-36c3a7595271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:59.568632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4e395-ef92-4842-90d2-36c3a7595271 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:59.568910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4e395-ef92-4842-90d2-36c3a7595271 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:59.569982 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4e395-ef92-4842-90d2-36c3a7595271 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:59.580719 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4e395-ef92-4842-90d2-36c3a7595271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-cda4e395-ef92-4842-90d2-36c3a7595271 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:36:59.587148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:59.588183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:59.589005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:59.589005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:59.589005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:59.589005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:59.589005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:36:59.589005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:36:59.589005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:36:59.589005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:36:59.589005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:36:59.589005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:36:59.589005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:36:59.589005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:59.589005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 263/509] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:36:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:59.589665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4e395-ef92-4842-90d2-36c3a7595271 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:59.607819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3595652-9db6-4d39-b22d-936f13f40cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:59.608515 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3595652-9db6-4d39-b22d-936f13f40cf3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:59.608795 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3595652-9db6-4d39-b22d-936f13f40cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:59.609062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3595652-9db6-4d39-b22d-936f13f40cf3 None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:59.609855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3595652-9db6-4d39-b22d-936f13f40cf3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:59.610161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3595652-9db6-4d39-b22d-936f13f40cf3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:59.611620 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3595652-9db6-4d39-b22d-936f13f40cf3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:59.624131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:59.652698 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 247/510] 10.222.0.86 () {56 vars in 1137 bytes} [Fri Aug 5 19:36:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:59.653557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3595652-9db6-4d39-b22d-936f13f40cf3 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:59.701195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4e395-ef92-4842-90d2-36c3a7595271 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:59.737022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c31924-6e3d-4256-b93f-c0cdcd3d9ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:59.737639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c31924-6e3d-4256-b93f-c0cdcd3d9ef9 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:59.737921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c31924-6e3d-4256-b93f-c0cdcd3d9ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:59.738191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c31924-6e3d-4256-b93f-c0cdcd3d9ef9 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:59.738915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c31924-6e3d-4256-b93f-c0cdcd3d9ef9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:59.739221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c31924-6e3d-4256-b93f-c0cdcd3d9ef9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:59.740607 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c31924-6e3d-4256-b93f-c0cdcd3d9ef9 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:59.753441 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 264/511] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:59.754992 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c31924-6e3d-4256-b93f-c0cdcd3d9ef9 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:59.764028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3595652-9db6-4d39-b22d-936f13f40cf3 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:59.799544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd555faf-b230-4ab4-94b9-58889eef6c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:59.800080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd555faf-b230-4ab4-94b9-58889eef6c9f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:59.800360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd555faf-b230-4ab4-94b9-58889eef6c9f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:59.800629 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd555faf-b230-4ab4-94b9-58889eef6c9f None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:59.801328 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd555faf-b230-4ab4-94b9-58889eef6c9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:59.801635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd555faf-b230-4ab4-94b9-58889eef6c9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:59.803060 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd555faf-b230-4ab4-94b9-58889eef6c9f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:59.815983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 248/512] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:59.817104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd555faf-b230-4ab4-94b9-58889eef6c9f None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:59.820565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:59.846520 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:36:59.860210 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c31924-6e3d-4256-b93f-c0cdcd3d9ef9 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:59.898877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a05a2-2cdd-40ca-8f8d-a10812d40018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:59.899615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a05a2-2cdd-40ca-8f8d-a10812d40018 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:59.899952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a05a2-2cdd-40ca-8f8d-a10812d40018 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:59.900292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a05a2-2cdd-40ca-8f8d-a10812d40018 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:59.909391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a05a2-2cdd-40ca-8f8d-a10812d40018 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:59.911266 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a05a2-2cdd-40ca-8f8d-a10812d40018 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-b13a05a2-2cdd-40ca-8f8d-a10812d40018 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:36:59.930027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:36:59.931085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:36:59.931985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:36:59.931985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:36:59.931985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:36:59.931985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:36:59.931985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:36:59.931985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:36:59.931985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:36:59.931985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:36:59.931985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:36:59.931985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:36:59.931985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 265/513] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:36:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:36:59.932551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd555faf-b230-4ab4-94b9-58889eef6c9f None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:36:59.932962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a05a2-2cdd-40ca-8f8d-a10812d40018 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:36:59.974457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3cf7e-e572-47cd-ab37-f9a8cea53ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:36:59.975672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3cf7e-e572-47cd-ab37-f9a8cea53ea1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:36:59.976014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3cf7e-e572-47cd-ab37-f9a8cea53ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:36:59.976583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3cf7e-e572-47cd-ab37-f9a8cea53ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:36:59.977658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3cf7e-e572-47cd-ab37-f9a8cea53ea1 None admin] RBAC: Adding query filter params (name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:36:59.978137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3cf7e-e572-47cd-ab37-f9a8cea53ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:36:59.980476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3cf7e-e572-47cd-ab37-f9a8cea53ea1 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:36:59.989124 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 249/514] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:36:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:36:59.989967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3cf7e-e572-47cd-ab37-f9a8cea53ea1 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:00.046847 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a05a2-2cdd-40ca-8f8d-a10812d40018 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:00.060484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:00.083980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256690a5-cc75-4769-82b5-437509d249c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:00.084597 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256690a5-cc75-4769-82b5-437509d249c1 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:00.085014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256690a5-cc75-4769-82b5-437509d249c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:00.085347 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256690a5-cc75-4769-82b5-437509d249c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:00.103449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3cf7e-e572-47cd-ab37-f9a8cea53ea1 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:00.104982 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:00.140395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9486b-28d4-4882-9686-0bd6ee7f9a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:00.141013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9486b-28d4-4882-9686-0bd6ee7f9a5c None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:00.141292 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9486b-28d4-4882-9686-0bd6ee7f9a5c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:00.141565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9486b-28d4-4882-9686-0bd6ee7f9a5c None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:00.142250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9486b-28d4-4882-9686-0bd6ee7f9a5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:00.143733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9486b-28d4-4882-9686-0bd6ee7f9a5c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:00.161240 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256690a5-cc75-4769-82b5-437509d249c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:00.163676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256690a5-cc75-4769-82b5-437509d249c1 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:00.166432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 250/515] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:36:59 2022] POST /identity/v3/users => generated 247 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:37:00.167421 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9486b-28d4-4882-9686-0bd6ee7f9a5c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:00.302346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9486b-28d4-4882-9686-0bd6ee7f9a5c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:00.307371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 266/516] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:36:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:37:00.309009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256690a5-cc75-4769-82b5-437509d249c1 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:00.329447 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:00.352708 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bd872-b2d5-4921-a46e-8d3d7c72da9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:00.353676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bd872-b2d5-4921-a46e-8d3d7c72da9c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:00.353993 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bd872-b2d5-4921-a46e-8d3d7c72da9c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:00.354272 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bd872-b2d5-4921-a46e-8d3d7c72da9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:00.364537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bd872-b2d5-4921-a46e-8d3d7c72da9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:00.365173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bd872-b2d5-4921-a46e-8d3d7c72da9c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:00.373893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-f78bd872-b2d5-4921-a46e-8d3d7c72da9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:00.380371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:00.381482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:00.383494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:00.383494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:00.383494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:00.383494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:00.383494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:00.383494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:00.383494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:00.383494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:00.383494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 251/517] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:00.383494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bd872-b2d5-4921-a46e-8d3d7c72da9c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:00.438570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256690a5-cc75-4769-82b5-437509d249c1 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:00.487784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c8edd-e440-44fc-aef2-acf3d85cd0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:00.488407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c8edd-e440-44fc-aef2-acf3d85cd0c4 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:00.488690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c8edd-e440-44fc-aef2-acf3d85cd0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:00.488961 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c8edd-e440-44fc-aef2-acf3d85cd0c4 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:00.489768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c8edd-e440-44fc-aef2-acf3d85cd0c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:00.490081 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c8edd-e440-44fc-aef2-acf3d85cd0c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:00.491761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c8edd-e440-44fc-aef2-acf3d85cd0c4 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:00.519962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 267/518] 10.222.0.86 () {56 vars in 1137 bytes} [Fri Aug 5 19:37:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:00.519962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c8edd-e440-44fc-aef2-acf3d85cd0c4 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:00.523756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bd872-b2d5-4921-a46e-8d3d7c72da9c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:00.574166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d92d9d-ce8c-4ca5-bad2-0e4a5598b42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:00.574867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d92d9d-ce8c-4ca5-bad2-0e4a5598b42e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:00.575148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d92d9d-ce8c-4ca5-bad2-0e4a5598b42e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:00.575417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d92d9d-ce8c-4ca5-bad2-0e4a5598b42e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:00.588027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d92d9d-ce8c-4ca5-bad2-0e4a5598b42e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:00.589439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d92d9d-ce8c-4ca5-bad2-0e4a5598b42e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-06d92d9d-ce8c-4ca5-bad2-0e4a5598b42e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:00.604503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:00.605533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:00.611144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:00.611144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:00.611144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:00.611144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:00.611144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 252/519] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:00.611144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d92d9d-ce8c-4ca5-bad2-0e4a5598b42e None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:00.645764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:00.660139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c8edd-e440-44fc-aef2-acf3d85cd0c4 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:00.669546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:00.706121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ddc46-6bbe-4172-8503-2df1a06ac3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:00.706920 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ddc46-6bbe-4172-8503-2df1a06ac3fe None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:00.707268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ddc46-6bbe-4172-8503-2df1a06ac3fe None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:00.707609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ddc46-6bbe-4172-8503-2df1a06ac3fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:00.719157 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ddc46-6bbe-4172-8503-2df1a06ac3fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:00.720740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ddc46-6bbe-4172-8503-2df1a06ac3fe None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:00.727423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d92d9d-ce8c-4ca5-bad2-0e4a5598b42e None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-ae9ddc46-6bbe-4172-8503-2df1a06ac3fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:00.739807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:00.740952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:00.742571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:00.742571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:00.742571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:00.742571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:00.747953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 268/520] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:00.747953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ddc46-6bbe-4172-8503-2df1a06ac3fe None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:00.779422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bdb69-8247-422a-9ba8-b9a54a4b1270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:00.780340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bdb69-8247-422a-9ba8-b9a54a4b1270 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:00.780666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bdb69-8247-422a-9ba8-b9a54a4b1270 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:00.780946 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bdb69-8247-422a-9ba8-b9a54a4b1270 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:00.781709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bdb69-8247-422a-9ba8-b9a54a4b1270 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:00.782081 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bdb69-8247-422a-9ba8-b9a54a4b1270 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:00.783646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bdb69-8247-422a-9ba8-b9a54a4b1270 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:00.805078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 253/521] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:00.805078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bdb69-8247-422a-9ba8-b9a54a4b1270 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:00.880836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ddc46-6bbe-4172-8503-2df1a06ac3fe None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:00.920881 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:00.929930 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010ff31-8461-4384-9419-e05cac0c7392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:00.931045 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010ff31-8461-4384-9419-e05cac0c7392 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:00.931535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010ff31-8461-4384-9419-e05cac0c7392 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:00.932012 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010ff31-8461-4384-9419-e05cac0c7392 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:00.933230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bdb69-8247-422a-9ba8-b9a54a4b1270 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:00.984833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:00.996976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee64df-1a09-4363-a269-c9c818fbb5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:00.998873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee64df-1a09-4363-a269-c9c818fbb5ef None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:00.999296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee64df-1a09-4363-a269-c9c818fbb5ef None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:00.999773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee64df-1a09-4363-a269-c9c818fbb5ef None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:01.000687 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee64df-1a09-4363-a269-c9c818fbb5ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:01.001179 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee64df-1a09-4363-a269-c9c818fbb5ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:01.002899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee64df-1a09-4363-a269-c9c818fbb5ef None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:01.028525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 254/522] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:01.028525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee64df-1a09-4363-a269-c9c818fbb5ef None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:01.058901 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010ff31-8461-4384-9419-e05cac0c7392 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:01.060474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010ff31-8461-4384-9419-e05cac0c7392 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:01.152459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee64df-1a09-4363-a269-c9c818fbb5ef None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:01.187270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 269/523] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:37:01.188284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010ff31-8461-4384-9419-e05cac0c7392 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:01.192653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d60042-4ab4-46b2-bb0a-8460fc5816b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:01.193287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d60042-4ab4-46b2-bb0a-8460fc5816b0 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:01.193570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d60042-4ab4-46b2-bb0a-8460fc5816b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:01.193845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d60042-4ab4-46b2-bb0a-8460fc5816b0 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:01.195303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d60042-4ab4-46b2-bb0a-8460fc5816b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:01.195612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d60042-4ab4-46b2-bb0a-8460fc5816b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:01.197028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d60042-4ab4-46b2-bb0a-8460fc5816b0 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:01.210063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 255/524] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:01.211765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d60042-4ab4-46b2-bb0a-8460fc5816b0 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:01.231597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:01.262799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:01.311221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010ff31-8461-4384-9419-e05cac0c7392 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:01.340492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d60042-4ab4-46b2-bb0a-8460fc5816b0 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:01.368496 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99948d92-f65d-4d2a-8d66-6113b6350ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:01.369372 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99948d92-f65d-4d2a-8d66-6113b6350ba5 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:01.369852 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99948d92-f65d-4d2a-8d66-6113b6350ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:01.370326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99948d92-f65d-4d2a-8d66-6113b6350ba5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:01.371408 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99948d92-f65d-4d2a-8d66-6113b6350ba5 None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=ab8f9d9cf4ca4e38ae6eb67e949b40fd) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:01.371727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99948d92-f65d-4d2a-8d66-6113b6350ba5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:01.373254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99948d92-f65d-4d2a-8d66-6113b6350ba5 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:01.390850 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 270/525] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:01 2022] GET /identity/v3/role_assignments?user.id=ab8f9d9cf4ca4e38ae6eb67e949b40fd&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:01.391869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99948d92-f65d-4d2a-8d66-6113b6350ba5 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:01.397113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593ee1f-fe78-4201-8c76-15803f04582f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:01.399455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593ee1f-fe78-4201-8c76-15803f04582f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:01.400257 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593ee1f-fe78-4201-8c76-15803f04582f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:01.400590 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593ee1f-fe78-4201-8c76-15803f04582f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:01.413505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593ee1f-fe78-4201-8c76-15803f04582f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:01.415149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593ee1f-fe78-4201-8c76-15803f04582f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-1593ee1f-fe78-4201-8c76-15803f04582f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:01.432393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:01.433600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:01.435021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:01.435021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:01.435021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:01.435021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:01.435021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:01.435021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:01.435021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:01.435021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:01.435021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:37:01.435021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:01.435021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 256/526] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:01.435815 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593ee1f-fe78-4201-8c76-15803f04582f None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:01.568722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99948d92-f65d-4d2a-8d66-6113b6350ba5 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:01.574839 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:01.600380 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:01.613579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593ee1f-fe78-4201-8c76-15803f04582f None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:01.615425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe23a2e-6ad1-49a8-8c83-5fbd790d6e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:01.618423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe23a2e-6ad1-49a8-8c83-5fbd790d6e31 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:01.619326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe23a2e-6ad1-49a8-8c83-5fbd790d6e31 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:01.619705 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe23a2e-6ad1-49a8-8c83-5fbd790d6e31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:01.634445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe23a2e-6ad1-49a8-8c83-5fbd790d6e31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:01.636227 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe23a2e-6ad1-49a8-8c83-5fbd790d6e31 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-cbe23a2e-6ad1-49a8-8c83-5fbd790d6e31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:01.652704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:01.653654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:01.654581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:01.654581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:01.654581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:01.654581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:01.654581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 271/527] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:01.655438 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe23a2e-6ad1-49a8-8c83-5fbd790d6e31 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:01.667490 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a75c8-d19f-443f-8775-4002f15b9464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:01.668072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a75c8-d19f-443f-8775-4002f15b9464 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:01.668355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a75c8-d19f-443f-8775-4002f15b9464 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:01.668627 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a75c8-d19f-443f-8775-4002f15b9464 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:01.677284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a75c8-d19f-443f-8775-4002f15b9464 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:01.680444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a75c8-d19f-443f-8775-4002f15b9464 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-ec4a75c8-d19f-443f-8775-4002f15b9464 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:01.696049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:01.697391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:01.698338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:01.698338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:01.698338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:01.698338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:01.698338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:01.698338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:01.698338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:01.698338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:01.698338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 257/528] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:01.699503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a75c8-d19f-443f-8775-4002f15b9464 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:01.776538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe23a2e-6ad1-49a8-8c83-5fbd790d6e31 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:01.815770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54858d-446c-49b9-a993-d1d5477111f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:01.816412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54858d-446c-49b9-a993-d1d5477111f4 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:01.816688 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54858d-446c-49b9-a993-d1d5477111f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:01.816952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54858d-446c-49b9-a993-d1d5477111f4 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:01.817768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54858d-446c-49b9-a993-d1d5477111f4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:01.818080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54858d-446c-49b9-a993-d1d5477111f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:01.820610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54858d-446c-49b9-a993-d1d5477111f4 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:01.827095 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a75c8-d19f-443f-8775-4002f15b9464 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:01.844932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 272/529] 10.222.0.86 () {56 vars in 1139 bytes} [Fri Aug 5 19:37:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:01.845931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54858d-446c-49b9-a993-d1d5477111f4 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:01.869207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:01.876964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211e845-11f2-4367-8374-eef1cb4d83fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:01.878057 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211e845-11f2-4367-8374-eef1cb4d83fb None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:01.878782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211e845-11f2-4367-8374-eef1cb4d83fb None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:01.879142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211e845-11f2-4367-8374-eef1cb4d83fb None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:01.879993 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211e845-11f2-4367-8374-eef1cb4d83fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:01.880410 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211e845-11f2-4367-8374-eef1cb4d83fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:01.883309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211e845-11f2-4367-8374-eef1cb4d83fb None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:01.892525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:01.903618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 258/530] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:01.905324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211e845-11f2-4367-8374-eef1cb4d83fb None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:02.001093 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54858d-446c-49b9-a993-d1d5477111f4 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:02.040366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d427cfc-8da4-40d2-8214-bb0b2c4dba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:02.040998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d427cfc-8da4-40d2-8214-bb0b2c4dba9b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:02.041276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d427cfc-8da4-40d2-8214-bb0b2c4dba9b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:02.041548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d427cfc-8da4-40d2-8214-bb0b2c4dba9b None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:02.042311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d427cfc-8da4-40d2-8214-bb0b2c4dba9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:02.042636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d427cfc-8da4-40d2-8214-bb0b2c4dba9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:02.044365 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d427cfc-8da4-40d2-8214-bb0b2c4dba9b None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:02.050611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211e845-11f2-4367-8374-eef1cb4d83fb None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:02.065132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 273/531] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:02.066021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d427cfc-8da4-40d2-8214-bb0b2c4dba9b None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:02.098183 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6b3f1-a5c4-40b9-b4e5-08519e32199e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:02.099241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6b3f1-a5c4-40b9-b4e5-08519e32199e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:02.099727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6b3f1-a5c4-40b9-b4e5-08519e32199e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:02.100210 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6b3f1-a5c4-40b9-b4e5-08519e32199e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:02.169340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:02.174071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:02.179002 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d427cfc-8da4-40d2-8214-bb0b2c4dba9b None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:02.183193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6b3f1-a5c4-40b9-b4e5-08519e32199e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:02.185331 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6b3f1-a5c4-40b9-b4e5-08519e32199e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:02.223312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec249ef-139c-402e-8cdc-c7f935a00c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:02.223957 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec249ef-139c-402e-8cdc-c7f935a00c34 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:02.224236 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec249ef-139c-402e-8cdc-c7f935a00c34 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:02.224507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec249ef-139c-402e-8cdc-c7f935a00c34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:02.233385 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec249ef-139c-402e-8cdc-c7f935a00c34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:02.234896 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec249ef-139c-402e-8cdc-c7f935a00c34 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-1ec249ef-139c-402e-8cdc-c7f935a00c34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:02.249035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:02.249968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:02.251321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:02.251321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:02.251321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:02.251321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:02.251321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:02.251321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:02.251321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:02.251321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:02.251321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 274/532] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:02.251321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec249ef-139c-402e-8cdc-c7f935a00c34 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:02.316089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 259/533] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:37:02.317356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6b3f1-a5c4-40b9-b4e5-08519e32199e None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:02.375160 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec249ef-139c-402e-8cdc-c7f935a00c34 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:02.420856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ff0ca-87cb-40af-baac-069aca5fed13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:02.421473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ff0ca-87cb-40af-baac-069aca5fed13 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:02.421749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ff0ca-87cb-40af-baac-069aca5fed13 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:02.422020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ff0ca-87cb-40af-baac-069aca5fed13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:02.422838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ff0ca-87cb-40af-baac-069aca5fed13 None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=afea7d279292424490beff0481ff1512) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:02.423149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ff0ca-87cb-40af-baac-069aca5fed13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:02.424606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ff0ca-87cb-40af-baac-069aca5fed13 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:02.430303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:02.438668 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 275/534] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:02 2022] GET /identity/v3/role_assignments?user.id=afea7d279292424490beff0481ff1512&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:02.438668 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ff0ca-87cb-40af-baac-069aca5fed13 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:02.441357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6b3f1-a5c4-40b9-b4e5-08519e32199e None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:02.469434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:02.490980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a2a10-dd0a-40ee-9c6e-c1e454beaa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:02.491870 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2a10-dd0a-40ee-9c6e-c1e454beaa34 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:02.492285 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2a10-dd0a-40ee-9c6e-c1e454beaa34 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:02.492702 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2a10-dd0a-40ee-9c6e-c1e454beaa34 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:02.493786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a2a10-dd0a-40ee-9c6e-c1e454beaa34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:02.494289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a2a10-dd0a-40ee-9c6e-c1e454beaa34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:02.496004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a2a10-dd0a-40ee-9c6e-c1e454beaa34 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:02.517544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 260/535] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:02.519763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbe45-8fe6-4c4b-9ae1-06c91d441cc8 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:02.520265 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbe45-8fe6-4c4b-9ae1-06c91d441cc8 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:02.520755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbe45-8fe6-4c4b-9ae1-06c91d441cc8 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:02.523402 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 261/536] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:02.524640 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdbe45-8fe6-4c4b-9ae1-06c91d441cc8 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:02.594958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ff0ca-87cb-40af-baac-069aca5fed13 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:02.636349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f70fb-a0bc-4e2b-b7d8-1da01905082f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:02.636989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f70fb-a0bc-4e2b-b7d8-1da01905082f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:02.637258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f70fb-a0bc-4e2b-b7d8-1da01905082f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:02.637532 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f70fb-a0bc-4e2b-b7d8-1da01905082f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:02.650098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f70fb-a0bc-4e2b-b7d8-1da01905082f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:02.651712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f70fb-a0bc-4e2b-b7d8-1da01905082f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:02.663861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdbe45-8fe6-4c4b-9ae1-06c91d441cc8 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-1d5f70fb-a0bc-4e2b-b7d8-1da01905082f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:02.668547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:02.669621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:02.670724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:02.670724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:02.670724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:02.670724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:02.670724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 276/537] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:02.671305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f2144-85c6-4943-9559-623ba250fc14 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:02.671305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f2144-85c6-4943-9559-623ba250fc14 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:02.671539 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f2144-85c6-4943-9559-623ba250fc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:02.692476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3e3f2144-85c6-4943-9559-623ba250fc14 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:02.693476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3e3f2144-85c6-4943-9559-623ba250fc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:02.709637 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff307e-c4d3-47d8-b8b5-6fc26b84935d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:02.711338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff307e-c4d3-47d8-b8b5-6fc26b84935d None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:02.711845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff307e-c4d3-47d8-b8b5-6fc26b84935d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:02.712293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff307e-c4d3-47d8-b8b5-6fc26b84935d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:02.713623 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff307e-c4d3-47d8-b8b5-6fc26b84935d None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=72d4459164844e718544d7fde333db76) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:02.713940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff307e-c4d3-47d8-b8b5-6fc26b84935d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:02.715503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff307e-c4d3-47d8-b8b5-6fc26b84935d None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:02.721959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 262/538] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:02 2022] GET /identity/v3/role_assignments?user.id=72d4459164844e718544d7fde333db76&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:02.722831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff307e-c4d3-47d8-b8b5-6fc26b84935d None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:02.746267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:02.778210 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 277/539] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:02.778765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:02.820146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f2144-85c6-4943-9559-623ba250fc14 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:02.856013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff307e-c4d3-47d8-b8b5-6fc26b84935d None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:02.899200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fe7b2-3ca6-4bdc-9c80-50a25034bc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:02.899837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fe7b2-3ca6-4bdc-9c80-50a25034bc94 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:02.900338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fe7b2-3ca6-4bdc-9c80-50a25034bc94 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:02.900619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fe7b2-3ca6-4bdc-9c80-50a25034bc94 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:02.901363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fe7b2-3ca6-4bdc-9c80-50a25034bc94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:02.901670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fe7b2-3ca6-4bdc-9c80-50a25034bc94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:02.903147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fe7b2-3ca6-4bdc-9c80-50a25034bc94 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:02.917955 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 263/540] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:02.923467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fe7b2-3ca6-4bdc-9c80-50a25034bc94 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:03.026984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f2144-85c6-4943-9559-623ba250fc14 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:03.088914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:03.129954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc0451-f443-4e35-8101-d196f68f4de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:03.130809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc0451-f443-4e35-8101-d196f68f4de4 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:03.131087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc0451-f443-4e35-8101-d196f68f4de4 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:03.131411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc0451-f443-4e35-8101-d196f68f4de4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:03.144697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fe7b2-3ca6-4bdc-9c80-50a25034bc94 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:03.156727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc0451-f443-4e35-8101-d196f68f4de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-a0dc0451-f443-4e35-8101-d196f68f4de4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:03.169824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:03.171009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:03.172019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:03.172019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:03.172019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:03.172019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:03.172019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:03.172019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:03.172019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:03.172019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:03.172019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:03.172019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:03.172019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:03.172019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:03.179658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 278/541] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:03.183823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc0451-f443-4e35-8101-d196f68f4de4 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:03.189013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333982b-ba57-401b-b957-d5203505b00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:03.189661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333982b-ba57-401b-b957-d5203505b00b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:03.189933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333982b-ba57-401b-b957-d5203505b00b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:03.190194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333982b-ba57-401b-b957-d5203505b00b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:03.203502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333982b-ba57-401b-b957-d5203505b00b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:03.204966 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333982b-ba57-401b-b957-d5203505b00b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-5333982b-ba57-401b-b957-d5203505b00b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:03.231055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:03.232149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:03.233216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:03.233216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:03.233216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:03.233216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:03.233216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:03.233216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:03.233216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:03.233216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:03.233701 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 264/542] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:03.240343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333982b-ba57-401b-b957-d5203505b00b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:03.324368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:03.387748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333982b-ba57-401b-b957-d5203505b00b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:03.428757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97cd87-4511-4636-a7b3-e5510be65830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:03.429783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97cd87-4511-4636-a7b3-e5510be65830 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:03.430069 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97cd87-4511-4636-a7b3-e5510be65830 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:03.430471 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97cd87-4511-4636-a7b3-e5510be65830 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:03.431354 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97cd87-4511-4636-a7b3-e5510be65830 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:03.431480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc0451-f443-4e35-8101-d196f68f4de4 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:03.431705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97cd87-4511-4636-a7b3-e5510be65830 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:03.433198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97cd87-4511-4636-a7b3-e5510be65830 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:03.447110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:03.447415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 265/543] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:03.453193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97cd87-4511-4636-a7b3-e5510be65830 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:03.518202 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f6936-1785-44e4-8315-7d62148c93c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:03.518839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f6936-1785-44e4-8315-7d62148c93c5 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:03.519116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f6936-1785-44e4-8315-7d62148c93c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:03.519380 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f6936-1785-44e4-8315-7d62148c93c5 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:03.520099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f6936-1785-44e4-8315-7d62148c93c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:03.520391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f6936-1785-44e4-8315-7d62148c93c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:03.521789 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f6936-1785-44e4-8315-7d62148c93c5 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:03.544928 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 279/544] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:03.550059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f6936-1785-44e4-8315-7d62148c93c5 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:03.653460 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97cd87-4511-4636-a7b3-e5510be65830 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:03.699687 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f5b2a-34ec-4fd1-a513-7f282f6eeaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:03.700321 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f5b2a-34ec-4fd1-a513-7f282f6eeaef None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:03.700596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f5b2a-34ec-4fd1-a513-7f282f6eeaef None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:03.700866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f5b2a-34ec-4fd1-a513-7f282f6eeaef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:03.701669 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f5b2a-34ec-4fd1-a513-7f282f6eeaef None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=80097afc3ce540e6a8060f696535e62c) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:03.701973 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f5b2a-34ec-4fd1-a513-7f282f6eeaef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:03.703437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f5b2a-34ec-4fd1-a513-7f282f6eeaef None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:03.723982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 266/545] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:03 2022] GET /identity/v3/role_assignments?user.id=80097afc3ce540e6a8060f696535e62c&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:03.751697 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f6936-1785-44e4-8315-7d62148c93c5 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:03.781234 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:03.807677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af725cde-0a95-46f2-b34c-6f751b92aca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:03.808843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af725cde-0a95-46f2-b34c-6f751b92aca7 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:03.809154 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af725cde-0a95-46f2-b34c-6f751b92aca7 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:03.809500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af725cde-0a95-46f2-b34c-6f751b92aca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:03.827690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af725cde-0a95-46f2-b34c-6f751b92aca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:03.829289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af725cde-0a95-46f2-b34c-6f751b92aca7 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-af725cde-0a95-46f2-b34c-6f751b92aca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:03.861724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:03.862680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:03.864185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:03.864185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:03.864185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:03.864185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:03.864185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 280/546] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:03.869121 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f5b2a-34ec-4fd1-a513-7f282f6eeaef None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:03.949685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:03.964564 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c114dfe-0e73-438f-898b-dd8e4528e1d6 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:03.964848 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c114dfe-0e73-438f-898b-dd8e4528e1d6 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:03.965121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c114dfe-0e73-438f-898b-dd8e4528e1d6 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:03.966780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 281/547] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:37:03.972065 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbdff7-dd06-4b99-8354-91b3bfc818d6 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:03.972338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbdff7-dd06-4b99-8354-91b3bfc818d6 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:03.972611 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbdff7-dd06-4b99-8354-91b3bfc818d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:04.005273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f5b2a-34ec-4fd1-a513-7f282f6eeaef None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:04.019476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-46dbdff7-dd06-4b99-8354-91b3bfc818d6 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:04.020700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-46dbdff7-dd06-4b99-8354-91b3bfc818d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:04.076575 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9066af-b75a-4328-8a78-c9db507d3469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:04.077248 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9066af-b75a-4328-8a78-c9db507d3469 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:04.077532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9066af-b75a-4328-8a78-c9db507d3469 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:04.077857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9066af-b75a-4328-8a78-c9db507d3469 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:04.078823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9066af-b75a-4328-8a78-c9db507d3469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:04.079132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9066af-b75a-4328-8a78-c9db507d3469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:04.080515 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9066af-b75a-4328-8a78-c9db507d3469 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:04.107412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 267/548] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:04.113617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9066af-b75a-4328-8a78-c9db507d3469 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:04.167058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 282/549] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:04.195825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbdff7-dd06-4b99-8354-91b3bfc818d6 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:04.301426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9066af-b75a-4328-8a78-c9db507d3469 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:04.339080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:04.341682 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbdff7-dd06-4b99-8354-91b3bfc818d6 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:04.343426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:04.359207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e1eee-6e02-42ad-92f6-bd71b6773655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:04.359864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e1eee-6e02-42ad-92f6-bd71b6773655 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:04.360142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e1eee-6e02-42ad-92f6-bd71b6773655 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:04.360412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e1eee-6e02-42ad-92f6-bd71b6773655 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:04.376279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e1eee-6e02-42ad-92f6-bd71b6773655 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:04.384833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff654d97-69c2-4ac4-9103-943262206ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:04.385515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff654d97-69c2-4ac4-9103-943262206ca2 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:04.385795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff654d97-69c2-4ac4-9103-943262206ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:04.386067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff654d97-69c2-4ac4-9103-943262206ca2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:04.386963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e1eee-6e02-42ad-92f6-bd71b6773655 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:04.399115 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff654d97-69c2-4ac4-9103-943262206ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-ff654d97-69c2-4ac4-9103-943262206ca2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:04.404559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:04.405654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:04.406461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:04.406461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:04.406461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:04.406461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:04.406461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:04.406461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:04.406461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:04.406461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:04.406461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:04.406461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:04.406461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:04.406461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:04.407741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 283/550] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:04.413273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff654d97-69c2-4ac4-9103-943262206ca2 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-896e1eee-6e02-42ad-92f6-bd71b6773655 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:04.414219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:04.415161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:04.416227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:04.416227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:04.416227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:04.416227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:04.416227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:04.416227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:04.416227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:04.416227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:04.416227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:37:04.416227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:04.416227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 268/551] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:37:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:04.430195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e1eee-6e02-42ad-92f6-bd71b6773655 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:04.576806 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e1eee-6e02-42ad-92f6-bd71b6773655 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:04.617862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d39781-0a79-4d6a-a4bd-c3711ff059ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:04.618493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d39781-0a79-4d6a-a4bd-c3711ff059ee None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:04.618811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d39781-0a79-4d6a-a4bd-c3711ff059ee None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:04.619062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d39781-0a79-4d6a-a4bd-c3711ff059ee None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:04.619884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d39781-0a79-4d6a-a4bd-c3711ff059ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:04.620186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d39781-0a79-4d6a-a4bd-c3711ff059ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:04.621650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d39781-0a79-4d6a-a4bd-c3711ff059ee None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:04.659820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 269/552] 10.222.0.86 () {56 vars in 1135 bytes} [Fri Aug 5 19:37:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:04.667391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d39781-0a79-4d6a-a4bd-c3711ff059ee None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:04.668750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff654d97-69c2-4ac4-9103-943262206ca2 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:04.745935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade7094-93c1-4088-a739-ba3ac1199f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:04.746986 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade7094-93c1-4088-a739-ba3ac1199f2c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:04.747531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade7094-93c1-4088-a739-ba3ac1199f2c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:04.748011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade7094-93c1-4088-a739-ba3ac1199f2c None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:04.748993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade7094-93c1-4088-a739-ba3ac1199f2c None admin] RBAC: Adding query filter params (name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:04.749509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade7094-93c1-4088-a739-ba3ac1199f2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:04.750361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:04.752995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade7094-93c1-4088-a739-ba3ac1199f2c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:04.766547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 284/553] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:04.776011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade7094-93c1-4088-a739-ba3ac1199f2c None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:04.806355 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:04.883317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d39781-0a79-4d6a-a4bd-c3711ff059ee None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:04.898649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade7094-93c1-4088-a739-ba3ac1199f2c None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:04.949625 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e896224-4935-4ef8-9a85-d16c762b4fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:04.950965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e896224-4935-4ef8-9a85-d16c762b4fb9 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:04.952028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e896224-4935-4ef8-9a85-d16c762b4fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:04.952874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e896224-4935-4ef8-9a85-d16c762b4fb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:04.958116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0d2f4-c3a5-485b-8ea6-5d4a939bc519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:04.960405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0d2f4-c3a5-485b-8ea6-5d4a939bc519 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:04.960918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0d2f4-c3a5-485b-8ea6-5d4a939bc519 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:04.961477 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0d2f4-c3a5-485b-8ea6-5d4a939bc519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:04.983425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e896224-4935-4ef8-9a85-d16c762b4fb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:04.986658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e896224-4935-4ef8-9a85-d16c762b4fb9 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-3e896224-4935-4ef8-9a85-d16c762b4fb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:05.019673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:05.020609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:05.021487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:05.021487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:05.021487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:05.021487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:05.025335 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 285/554] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:05.031661 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e896224-4935-4ef8-9a85-d16c762b4fb9 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:05.091233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0d2f4-c3a5-485b-8ea6-5d4a939bc519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:05.093054 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0d2f4-c3a5-485b-8ea6-5d4a939bc519 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:05.146306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:05.150644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:05.183908 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e896224-4935-4ef8-9a85-d16c762b4fb9 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:05.228803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 270/555] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:37:05.231072 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6999d3a-7be3-4998-bb3e-7dc87611046c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:05.231795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6999d3a-7be3-4998-bb3e-7dc87611046c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:05.232180 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6999d3a-7be3-4998-bb3e-7dc87611046c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:05.232540 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a0509-ca88-487f-976e-a15aa9871232 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:05.232711 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6999d3a-7be3-4998-bb3e-7dc87611046c None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:05.233242 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a0509-ca88-487f-976e-a15aa9871232 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:05.233546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6999d3a-7be3-4998-bb3e-7dc87611046c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:05.234062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a0509-ca88-487f-976e-a15aa9871232 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:05.234329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6999d3a-7be3-4998-bb3e-7dc87611046c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:05.235849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6999d3a-7be3-4998-bb3e-7dc87611046c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:05.242537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 271/556] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:05.243805 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a0509-ca88-487f-976e-a15aa9871232 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:05.251393 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 286/557] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:05.252912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef97246-9156-4f89-911c-82769fafe041 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:05.253296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef97246-9156-4f89-911c-82769fafe041 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:05.253669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef97246-9156-4f89-911c-82769fafe041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:05.274761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-4ef97246-9156-4f89-911c-82769fafe041 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:05.276713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4ef97246-9156-4f89-911c-82769fafe041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:05.365288 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 287/558] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:05.367150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf18ff-6722-4d15-b962-71b240c15350 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:05.367630 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf18ff-6722-4d15-b962-71b240c15350 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:05.367831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf18ff-6722-4d15-b962-71b240c15350 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:05.373249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 288/559] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:05.373249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf18ff-6722-4d15-b962-71b240c15350 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:05.377514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a0509-ca88-487f-976e-a15aa9871232 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:05.418158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bef31-4a97-4ed4-81c0-6615d39be90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:05.419147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bef31-4a97-4ed4-81c0-6615d39be90f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:05.419514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bef31-4a97-4ed4-81c0-6615d39be90f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:05.419875 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bef31-4a97-4ed4-81c0-6615d39be90f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:05.426900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:05.434317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bef31-4a97-4ed4-81c0-6615d39be90f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:05.435972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bef31-4a97-4ed4-81c0-6615d39be90f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-af8bef31-4a97-4ed4-81c0-6615d39be90f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:05.456457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:05.457609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:05.458703 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:05.458703 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:05.458703 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:05.458703 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:05.458703 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 272/560] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:05.464755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92d198-587d-49c0-9661-109fd7f0eff3 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:05.464755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92d198-587d-49c0-9661-109fd7f0eff3 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:05.464755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92d198-587d-49c0-9661-109fd7f0eff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:05.470258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:05.487406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-4c92d198-587d-49c0-9661-109fd7f0eff3 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:05.488595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-4c92d198-587d-49c0-9661-109fd7f0eff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:05.528271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf18ff-6722-4d15-b962-71b240c15350 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:05.575036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b94228-a9d3-4758-ba6d-5bda49388a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:05.575872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b94228-a9d3-4758-ba6d-5bda49388a70 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:05.576252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b94228-a9d3-4758-ba6d-5bda49388a70 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:05.576621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b94228-a9d3-4758-ba6d-5bda49388a70 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:05.585190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b94228-a9d3-4758-ba6d-5bda49388a70 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:05.587063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b94228-a9d3-4758-ba6d-5bda49388a70 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:05.587224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 273/561] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:05.587859 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92d198-587d-49c0-9661-109fd7f0eff3 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-f0b94228-a9d3-4758-ba6d-5bda49388a70 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:05.605252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:05.606555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:05.607856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:05.607856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:05.607856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:05.607856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:05.607856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:05.607856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:05.607856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:05.607856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:05.607856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:37:05.607856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:05.610198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 289/562] 10.222.0.86 () {56 vars in 1093 bytes} [Fri Aug 5 19:37:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:05.611212 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b94228-a9d3-4758-ba6d-5bda49388a70 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:05.731504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92d198-587d-49c0-9661-109fd7f0eff3 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:05.732708 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b94228-a9d3-4758-ba6d-5bda49388a70 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:05.756147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:05.766110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:05.777487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f526f4-b97e-44c9-ba31-6c10edd63988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:05.777744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15267d-38b7-4d2a-a0a9-a0ac6837ab6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:05.778105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f526f4-b97e-44c9-ba31-6c10edd63988 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:05.778381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f526f4-b97e-44c9-ba31-6c10edd63988 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:05.778484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15267d-38b7-4d2a-a0a9-a0ac6837ab6b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:05.778672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f526f4-b97e-44c9-ba31-6c10edd63988 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:05.778783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15267d-38b7-4d2a-a0a9-a0ac6837ab6b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:05.779098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15267d-38b7-4d2a-a0a9-a0ac6837ab6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:05.779369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f526f4-b97e-44c9-ba31-6c10edd63988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:05.779721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f526f4-b97e-44c9-ba31-6c10edd63988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:05.781025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f526f4-b97e-44c9-ba31-6c10edd63988 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:05.793374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15267d-38b7-4d2a-a0a9-a0ac6837ab6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:05.795785 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 290/563] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:05.796795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f526f4-b97e-44c9-ba31-6c10edd63988 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-6c15267d-38b7-4d2a-a0a9-a0ac6837ab6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:05.802908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:05.804111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:05.805110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:05.805110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:05.805110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:05.805110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:05.805110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:05.805110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:05.805110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:05.805110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:05.805110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:05.805110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:05.805110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:05.805110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:05.805110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 274/564] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:05.805837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15267d-38b7-4d2a-a0a9-a0ac6837ab6b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:05.926210 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15267d-38b7-4d2a-a0a9-a0ac6837ab6b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:05.927416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f526f4-b97e-44c9-ba31-6c10edd63988 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:05.965711 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e845a2-1a39-40d3-a495-496fb4d29019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:05.966351 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e845a2-1a39-40d3-a495-496fb4d29019 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:05.966649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e845a2-1a39-40d3-a495-496fb4d29019 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:05.966829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbb0b4-d53c-452f-adf4-e8aa433093ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:05.966956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e845a2-1a39-40d3-a495-496fb4d29019 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:05.967451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbb0b4-d53c-452f-adf4-e8aa433093ef None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:05.967731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbb0b4-d53c-452f-adf4-e8aa433093ef None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:05.968004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbb0b4-d53c-452f-adf4-e8aa433093ef None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:05.968809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbb0b4-d53c-452f-adf4-e8aa433093ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:05.969116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbb0b4-d53c-452f-adf4-e8aa433093ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:05.970547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbb0b4-d53c-452f-adf4-e8aa433093ef None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:05.982929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e845a2-1a39-40d3-a495-496fb4d29019 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-82e845a2-1a39-40d3-a495-496fb4d29019 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:05.988144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:05.989089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:05.995615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:05.995615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:05.995615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:05.995615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:05.995615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:05.995615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:05.995615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:05.995615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:05.995615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:05.995615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:05.995615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:05.995615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:05.996249 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 275/565] 10.222.0.86 () {56 vars in 1133 bytes} [Fri Aug 5 19:37:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:05.996338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 291/566] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:05.996338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e845a2-1a39-40d3-a495-496fb4d29019 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:05.996475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbb0b4-d53c-452f-adf4-e8aa433093ef None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:06.040204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:06.056519 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:06.113800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbb0b4-d53c-452f-adf4-e8aa433093ef None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:06.126418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e845a2-1a39-40d3-a495-496fb4d29019 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:06.158709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080eb70-8c02-4c8b-a74e-214025612ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:06.159359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080eb70-8c02-4c8b-a74e-214025612ee9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:06.159637 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080eb70-8c02-4c8b-a74e-214025612ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:06.159904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080eb70-8c02-4c8b-a74e-214025612ee9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:06.165203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4b405-f627-4d79-b280-08967724bb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:06.165825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b405-f627-4d79-b280-08967724bb4f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:06.166112 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b405-f627-4d79-b280-08967724bb4f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:06.166382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b405-f627-4d79-b280-08967724bb4f None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:06.167144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4b405-f627-4d79-b280-08967724bb4f None admin] RBAC: Adding query filter params (name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:06.167440 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4b405-f627-4d79-b280-08967724bb4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:06.167874 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080eb70-8c02-4c8b-a74e-214025612ee9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:06.168804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4b405-f627-4d79-b280-08967724bb4f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:06.169310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080eb70-8c02-4c8b-a74e-214025612ee9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:06.177404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 292/567] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:06.178216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4b405-f627-4d79-b280-08967724bb4f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-2080eb70-8c02-4c8b-a74e-214025612ee9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:06.182749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:06.183754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:06.184689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:06.184689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:06.184689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:06.184689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:06.184689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:06.184689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:06.184689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:06.184689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:06.184689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 276/568] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:06.184689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080eb70-8c02-4c8b-a74e-214025612ee9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:06.287088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4b405-f627-4d79-b280-08967724bb4f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:06.291895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:06.296195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080eb70-8c02-4c8b-a74e-214025612ee9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:06.298076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:06.327021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd04fb0-7a25-40fe-a925-70f092779795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:06.327634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd04fb0-7a25-40fe-a925-70f092779795 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:06.327908 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd04fb0-7a25-40fe-a925-70f092779795 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:06.328175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd04fb0-7a25-40fe-a925-70f092779795 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:06.328887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd04fb0-7a25-40fe-a925-70f092779795 None admin] RBAC: Adding query filter params (name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:06.329192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd04fb0-7a25-40fe-a925-70f092779795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:06.330560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd04fb0-7a25-40fe-a925-70f092779795 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:06.336502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a77bf-8e51-4d57-8635-802f12ed13b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:06.337144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a77bf-8e51-4d57-8635-802f12ed13b2 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:06.337422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a77bf-8e51-4d57-8635-802f12ed13b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:06.337693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a77bf-8e51-4d57-8635-802f12ed13b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:06.343531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 293/569] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:06.345569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8498da-ad24-48fb-bddc-0fde7063dbd1 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:06.346070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8498da-ad24-48fb-bddc-0fde7063dbd1 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:06.346547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8498da-ad24-48fb-bddc-0fde7063dbd1 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:06.348721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 294/570] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:06.349364 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8498da-ad24-48fb-bddc-0fde7063dbd1 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:06.352944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a77bf-8e51-4d57-8635-802f12ed13b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:06.354474 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a77bf-8e51-4d57-8635-802f12ed13b2 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-a31a77bf-8e51-4d57-8635-802f12ed13b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:06.369658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:06.370724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:06.371753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:06.371753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:06.371753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:06.371753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:06.371753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 277/571] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:06.372160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a77bf-8e51-4d57-8635-802f12ed13b2 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:06.462916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8498da-ad24-48fb-bddc-0fde7063dbd1 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:06.487740 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a77bf-8e51-4d57-8635-802f12ed13b2 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:06.499397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238029f-6317-4a9b-8cf3-5eb9d948a218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:06.501141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238029f-6317-4a9b-8cf3-5eb9d948a218 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:06.501141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238029f-6317-4a9b-8cf3-5eb9d948a218 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:06.501141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238029f-6317-4a9b-8cf3-5eb9d948a218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:06.512471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238029f-6317-4a9b-8cf3-5eb9d948a218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:06.513936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238029f-6317-4a9b-8cf3-5eb9d948a218 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:06.526397 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4b253-3f08-4cde-b85c-9894783fc8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:06.527098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4b253-3f08-4cde-b85c-9894783fc8fb None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:06.527327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4b253-3f08-4cde-b85c-9894783fc8fb None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:06.527598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4b253-3f08-4cde-b85c-9894783fc8fb None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:06.528346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4b253-3f08-4cde-b85c-9894783fc8fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:06.528650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4b253-3f08-4cde-b85c-9894783fc8fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-9238029f-6317-4a9b-8cf3-5eb9d948a218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:06.529944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:06.535656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:06.536796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:06.536796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:06.536796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:06.536796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:06.537064 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4b253-3f08-4cde-b85c-9894783fc8fb None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:06.537207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 295/572] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:06.537207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238029f-6317-4a9b-8cf3-5eb9d948a218 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:06.544213 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 278/573] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:06.545480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac55e34-c0af-47a4-9d03-3a2d88eaf081 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:06.545761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac55e34-c0af-47a4-9d03-3a2d88eaf081 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:06.546028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac55e34-c0af-47a4-9d03-3a2d88eaf081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:06.559854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:06.564242 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-5ac55e34-c0af-47a4-9d03-3a2d88eaf081 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:06.565489 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5ac55e34-c0af-47a4-9d03-3a2d88eaf081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:06.568464 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:06.650706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 279/574] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:06.651655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac55e34-c0af-47a4-9d03-3a2d88eaf081 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:06.668354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238029f-6317-4a9b-8cf3-5eb9d948a218 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:06.714361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d56c3c-5693-4e3a-b4ea-15cc3ce101fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:06.715032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d56c3c-5693-4e3a-b4ea-15cc3ce101fe None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:06.715305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d56c3c-5693-4e3a-b4ea-15cc3ce101fe None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:06.715575 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d56c3c-5693-4e3a-b4ea-15cc3ce101fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:06.726534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d56c3c-5693-4e3a-b4ea-15cc3ce101fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:06.728091 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d56c3c-5693-4e3a-b4ea-15cc3ce101fe None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-32d56c3c-5693-4e3a-b4ea-15cc3ce101fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:06.743959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:06.745038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:06.746121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:06.746121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:06.746121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:06.746121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:06.746121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 296/575] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:06.746462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d56c3c-5693-4e3a-b4ea-15cc3ce101fe None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:06.769683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac55e34-c0af-47a4-9d03-3a2d88eaf081 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:06.805197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc865762-a6f1-49d6-8a4d-3fb51b7a3622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:06.805731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc865762-a6f1-49d6-8a4d-3fb51b7a3622 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:06.806010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc865762-a6f1-49d6-8a4d-3fb51b7a3622 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:06.806319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc865762-a6f1-49d6-8a4d-3fb51b7a3622 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:06.807057 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc865762-a6f1-49d6-8a4d-3fb51b7a3622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:06.807360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc865762-a6f1-49d6-8a4d-3fb51b7a3622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:06.808763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc865762-a6f1-49d6-8a4d-3fb51b7a3622 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:06.817641 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:06.825775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 280/576] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:06.825775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc865762-a6f1-49d6-8a4d-3fb51b7a3622 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:06.840943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:06.869791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d56c3c-5693-4e3a-b4ea-15cc3ce101fe None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:06.914848 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f810e-3900-4890-8251-0cb1e4717c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:06.915468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f810e-3900-4890-8251-0cb1e4717c00 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:06.915747 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f810e-3900-4890-8251-0cb1e4717c00 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:06.916018 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f810e-3900-4890-8251-0cb1e4717c00 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:06.916725 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f810e-3900-4890-8251-0cb1e4717c00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:06.917027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f810e-3900-4890-8251-0cb1e4717c00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:06.918412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f810e-3900-4890-8251-0cb1e4717c00 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:06.931119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 297/577] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:06.931958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f810e-3900-4890-8251-0cb1e4717c00 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:06.932516 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc865762-a6f1-49d6-8a4d-3fb51b7a3622 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:06.971016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d838a09-5b08-4f1d-976c-5d4714630098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:06.971630 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d838a09-5b08-4f1d-976c-5d4714630098 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:06.971915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d838a09-5b08-4f1d-976c-5d4714630098 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:06.972187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d838a09-5b08-4f1d-976c-5d4714630098 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:06.972963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d838a09-5b08-4f1d-976c-5d4714630098 None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=e8adb0fd1fbc4268b70f3936036e550b) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:06.973269 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d838a09-5b08-4f1d-976c-5d4714630098 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:06.974734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d838a09-5b08-4f1d-976c-5d4714630098 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:06.980325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 281/578] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:06 2022] GET /identity/v3/role_assignments?user.id=e8adb0fd1fbc4268b70f3936036e550b&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:06.981217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d838a09-5b08-4f1d-976c-5d4714630098 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:07.045154 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f810e-3900-4890-8251-0cb1e4717c00 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:07.068546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:07.090554 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edd927-5af6-47b0-86d3-f5e3ee76935a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:07.091530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edd927-5af6-47b0-86d3-f5e3ee76935a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:07.091970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edd927-5af6-47b0-86d3-f5e3ee76935a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:07.092387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edd927-5af6-47b0-86d3-f5e3ee76935a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:07.094749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:07.107966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edd927-5af6-47b0-86d3-f5e3ee76935a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-d7edd927-5af6-47b0-86d3-f5e3ee76935a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:07.113345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:07.114552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:07.116328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:07.116328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:07.116328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:07.116328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:07.116328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:07.116328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:07.116328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:07.116328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:07.116328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:07.116328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:07.116328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:07.116328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:07.117328 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d838a09-5b08-4f1d-976c-5d4714630098 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:07.120138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 298/579] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:07.121089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edd927-5af6-47b0-86d3-f5e3ee76935a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:07.166260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8de22-2c2b-4b05-a9c8-0ee00980b31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:07.166945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8de22-2c2b-4b05-a9c8-0ee00980b31d None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:07.167213 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8de22-2c2b-4b05-a9c8-0ee00980b31d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:07.167537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8de22-2c2b-4b05-a9c8-0ee00980b31d None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:07.168233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8de22-2c2b-4b05-a9c8-0ee00980b31d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:07.168560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8de22-2c2b-4b05-a9c8-0ee00980b31d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:07.169983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8de22-2c2b-4b05-a9c8-0ee00980b31d None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:07.184776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 282/580] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:07.185927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8de22-2c2b-4b05-a9c8-0ee00980b31d None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:07.259438 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edd927-5af6-47b0-86d3-f5e3ee76935a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:07.302454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8de22-2c2b-4b05-a9c8-0ee00980b31d None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:07.316603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f63a29-24a8-49fd-93ac-a499e7829dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:07.317461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f63a29-24a8-49fd-93ac-a499e7829dda None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:07.317897 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f63a29-24a8-49fd-93ac-a499e7829dda None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:07.318279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f63a29-24a8-49fd-93ac-a499e7829dda None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:07.326812 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f63a29-24a8-49fd-93ac-a499e7829dda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:07.328314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f63a29-24a8-49fd-93ac-a499e7829dda None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:07.333686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-46f63a29-24a8-49fd-93ac-a499e7829dda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:07.341956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:07.343995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:07.346059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:07.346059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:07.346059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:07.346059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:07.346059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:07.346059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:07.346059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:07.346059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:07.346059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 299/581] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:07.346059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f63a29-24a8-49fd-93ac-a499e7829dda None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:07.354435 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a6f86-6eee-4fcd-b11b-b40048d548e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:07.355247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a6f86-6eee-4fcd-b11b-b40048d548e4 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:07.355706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a6f86-6eee-4fcd-b11b-b40048d548e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:07.355989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a6f86-6eee-4fcd-b11b-b40048d548e4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:07.363783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a6f86-6eee-4fcd-b11b-b40048d548e4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:07.365363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a6f86-6eee-4fcd-b11b-b40048d548e4 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-2d1a6f86-6eee-4fcd-b11b-b40048d548e4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:07.378810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:07.380850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:07.382116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:07.382116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:07.382116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:07.382116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:07.382116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:07.382116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:07.382116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:07.382116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:07.382116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:37:07.382116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:07.382116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 283/582] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:37:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:07.382116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a6f86-6eee-4fcd-b11b-b40048d548e4 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:07.424406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:07.484016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f63a29-24a8-49fd-93ac-a499e7829dda None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:07.512453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a6f86-6eee-4fcd-b11b-b40048d548e4 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:07.533318 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907f864-f0ee-4de1-8474-28927062c639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:07.533948 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907f864-f0ee-4de1-8474-28927062c639 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:07.534222 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907f864-f0ee-4de1-8474-28927062c639 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:07.534496 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907f864-f0ee-4de1-8474-28927062c639 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:07.535251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907f864-f0ee-4de1-8474-28927062c639 None admin] RBAC: Adding query filter params (name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:07.535549 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907f864-f0ee-4de1-8474-28927062c639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:07.536941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907f864-f0ee-4de1-8474-28927062c639 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:07.546337 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 300/583] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:07.547297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907f864-f0ee-4de1-8474-28927062c639 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:07.554247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c9c32-0d37-4de9-ab1f-e7276614bca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:07.555091 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c9c32-0d37-4de9-ab1f-e7276614bca3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:07.555473 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c9c32-0d37-4de9-ab1f-e7276614bca3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:07.555841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c9c32-0d37-4de9-ab1f-e7276614bca3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:07.566848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c9c32-0d37-4de9-ab1f-e7276614bca3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:07.567513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c9c32-0d37-4de9-ab1f-e7276614bca3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-f36c9c32-0d37-4de9-ab1f-e7276614bca3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:07.580538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:07.581510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:07.582449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:07.582449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:07.582449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:07.582449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:07.582449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:07.582449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:07.582449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:07.582449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:07.582449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:37:07.582449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:07.582449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 284/584] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:37:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:07.582449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c9c32-0d37-4de9-ab1f-e7276614bca3 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:07.642247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:07.675037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907f864-f0ee-4de1-8474-28927062c639 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:07.695703 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:07.706996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c9c32-0d37-4de9-ab1f-e7276614bca3 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:07.722260 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8ecfe-7d66-4859-ae23-49eaea3000ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:07.723647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8ecfe-7d66-4859-ae23-49eaea3000ac None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:07.724053 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8ecfe-7d66-4859-ae23-49eaea3000ac None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:07.724453 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8ecfe-7d66-4859-ae23-49eaea3000ac None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:07.725329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8ecfe-7d66-4859-ae23-49eaea3000ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:07.725714 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8ecfe-7d66-4859-ae23-49eaea3000ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:07.727502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8ecfe-7d66-4859-ae23-49eaea3000ac None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:07.750434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 301/585] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:07.750434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8ecfe-7d66-4859-ae23-49eaea3000ac None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:07.754531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506e8ae-9c40-4682-955a-ff26ba4ce6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:07.755274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506e8ae-9c40-4682-955a-ff26ba4ce6de None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:07.755553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506e8ae-9c40-4682-955a-ff26ba4ce6de None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:07.756062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506e8ae-9c40-4682-955a-ff26ba4ce6de None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:07.757477 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506e8ae-9c40-4682-955a-ff26ba4ce6de None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:07.757785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506e8ae-9c40-4682-955a-ff26ba4ce6de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:07.759389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506e8ae-9c40-4682-955a-ff26ba4ce6de None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:07.785799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 285/586] 10.222.0.86 () {56 vars in 1137 bytes} [Fri Aug 5 19:37:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:07.791467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506e8ae-9c40-4682-955a-ff26ba4ce6de None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:07.871952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8ecfe-7d66-4859-ae23-49eaea3000ac None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:07.908267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506e8ae-9c40-4682-955a-ff26ba4ce6de None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:07.910074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:07.913652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2c81e-2b65-454c-9d45-466789d4ef68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:07.914317 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2c81e-2b65-454c-9d45-466789d4ef68 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:07.914599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2c81e-2b65-454c-9d45-466789d4ef68 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:07.914902 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2c81e-2b65-454c-9d45-466789d4ef68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:07.926962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2c81e-2b65-454c-9d45-466789d4ef68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:07.928939 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2c81e-2b65-454c-9d45-466789d4ef68 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-5cf2c81e-2b65-454c-9d45-466789d4ef68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:07.945883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:07.946909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:07.947818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:07.947818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:07.947818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:07.947818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:07.947818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 302/587] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:07.948111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2c81e-2b65-454c-9d45-466789d4ef68 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:07.959172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d90ff-f9a6-4335-a3f8-ef8f33d0664b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:07.959843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d90ff-f9a6-4335-a3f8-ef8f33d0664b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:07.960316 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d90ff-f9a6-4335-a3f8-ef8f33d0664b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:07.960768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d90ff-f9a6-4335-a3f8-ef8f33d0664b None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:07.961687 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d90ff-f9a6-4335-a3f8-ef8f33d0664b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:07.961999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d90ff-f9a6-4335-a3f8-ef8f33d0664b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:07.963476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d90ff-f9a6-4335-a3f8-ef8f33d0664b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:07.987514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 286/588] 10.222.0.86 () {56 vars in 1137 bytes} [Fri Aug 5 19:37:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:07.987514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d90ff-f9a6-4335-a3f8-ef8f33d0664b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:07.995591 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:08.085238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2c81e-2b65-454c-9d45-466789d4ef68 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:08.105128 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d90ff-f9a6-4335-a3f8-ef8f33d0664b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:08.125054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64788b9-14ed-4099-a3bf-e0fd49811673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:08.125657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64788b9-14ed-4099-a3bf-e0fd49811673 None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:08.125936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64788b9-14ed-4099-a3bf-e0fd49811673 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:08.126208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64788b9-14ed-4099-a3bf-e0fd49811673 None admin] PATH_INFO: `/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/ab8f9d9cf4ca4e38ae6eb67e949b40fd/roles/02001b18565340ce984343c5d36daaf9` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:08.146265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64788b9-14ed-4099-a3bf-e0fd49811673 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c73d2cfaa1c47829ca6936186d2bb26, user_id=ab8f9d9cf4ca4e38ae6eb67e949b40fd, role_id=02001b18565340ce984343c5d36daaf9)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:08.146853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c864c-2bf7-4f9d-8778-7200b0693dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:08.147892 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64788b9-14ed-4099-a3bf-e0fd49811673 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:08.147996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c864c-2bf7-4f9d-8778-7200b0693dc5 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:08.148382 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c864c-2bf7-4f9d-8778-7200b0693dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:08.148865 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c864c-2bf7-4f9d-8778-7200b0693dc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:08.164354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 303/589] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:37:07 2022] PUT /identity/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/ab8f9d9cf4ca4e38ae6eb67e949b40fd/roles/02001b18565340ce984343c5d36daaf9 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:37:08.164946 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64788b9-14ed-4099-a3bf-e0fd49811673 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:08.182440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c864c-2bf7-4f9d-8778-7200b0693dc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:08.184333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c864c-2bf7-4f9d-8778-7200b0693dc5 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-7c6c864c-2bf7-4f9d-8778-7200b0693dc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:08.214939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:08.217735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:08.222214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:08.222214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:08.222214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:08.222214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:08.223043 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 287/590] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:08.224136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c864c-2bf7-4f9d-8778-7200b0693dc5 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:08.245204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:08.330419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:08.349827 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c864c-2bf7-4f9d-8778-7200b0693dc5 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:08.376924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64788b9-14ed-4099-a3bf-e0fd49811673 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:08.389929 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0350a4-e834-4581-8cbc-72eabec35b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:08.390579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0350a4-e834-4581-8cbc-72eabec35b70 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:08.390890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0350a4-e834-4581-8cbc-72eabec35b70 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:08.391233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0350a4-e834-4581-8cbc-72eabec35b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:08.405809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0350a4-e834-4581-8cbc-72eabec35b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:08.407246 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0350a4-e834-4581-8cbc-72eabec35b70 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-fd0350a4-e834-4581-8cbc-72eabec35b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:08.423843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:08.424820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:08.426305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:08.426305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:08.426305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:08.426305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:08.426305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 288/591] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:08.426305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0350a4-e834-4581-8cbc-72eabec35b70 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:08.441353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47928e-7603-477c-a762-034ee75f3ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:08.442051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47928e-7603-477c-a762-034ee75f3ebc None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:08.442331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47928e-7603-477c-a762-034ee75f3ebc None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:08.442603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47928e-7603-477c-a762-034ee75f3ebc None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:08.443334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47928e-7603-477c-a762-034ee75f3ebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:08.443632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47928e-7603-477c-a762-034ee75f3ebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:08.446024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47928e-7603-477c-a762-034ee75f3ebc None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:08.463480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 304/592] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:08.464403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47928e-7603-477c-a762-034ee75f3ebc None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:08.545900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:08.592754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0350a4-e834-4581-8cbc-72eabec35b70 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:08.613434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47928e-7603-477c-a762-034ee75f3ebc None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:08.638615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9bb80-a8d5-4836-9333-bcdb57e1e7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:08.639528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9bb80-a8d5-4836-9333-bcdb57e1e7a8 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:08.640059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9bb80-a8d5-4836-9333-bcdb57e1e7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:08.640527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9bb80-a8d5-4836-9333-bcdb57e1e7a8 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:08.641843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9bb80-a8d5-4836-9333-bcdb57e1e7a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:08.642377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9bb80-a8d5-4836-9333-bcdb57e1e7a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:08.644166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9bb80-a8d5-4836-9333-bcdb57e1e7a8 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:08.659914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267080eb-0327-4e6f-8a16-b928a6099f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:08.660813 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267080eb-0327-4e6f-8a16-b928a6099f95 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:08.661233 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267080eb-0327-4e6f-8a16-b928a6099f95 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:08.661654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267080eb-0327-4e6f-8a16-b928a6099f95 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:08.662491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267080eb-0327-4e6f-8a16-b928a6099f95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:08.663603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267080eb-0327-4e6f-8a16-b928a6099f95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:08.665153 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267080eb-0327-4e6f-8a16-b928a6099f95 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:08.665447 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 289/593] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:08.666764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9bb80-a8d5-4836-9333-bcdb57e1e7a8 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:08.674961 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:08.688394 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 305/594] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:08.688394 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267080eb-0327-4e6f-8a16-b928a6099f95 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:08.797509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9bb80-a8d5-4836-9333-bcdb57e1e7a8 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:08.829533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:08.832786 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267080eb-0327-4e6f-8a16-b928a6099f95 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:08.843509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68476e50-c6c5-48fa-8cd9-9f6bb84f48ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:08.844460 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68476e50-c6c5-48fa-8cd9-9f6bb84f48ab None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:08.844704 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68476e50-c6c5-48fa-8cd9-9f6bb84f48ab None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:08.845045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68476e50-c6c5-48fa-8cd9-9f6bb84f48ab None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:08.856648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68476e50-c6c5-48fa-8cd9-9f6bb84f48ab None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:08.858194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68476e50-c6c5-48fa-8cd9-9f6bb84f48ab None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-68476e50-c6c5-48fa-8cd9-9f6bb84f48ab None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:08.871737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:08.872741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:08.873663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:08.873663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:08.873663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:08.873663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:08.873663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:08.873663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:08.873663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:08.873663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:08.873663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:37:08.873663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:08.873663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 290/595] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:08.874262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68476e50-c6c5-48fa-8cd9-9f6bb84f48ab None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:08.883026 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7ebcd-ea66-4c2c-9345-a770a31543b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:08.883864 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7ebcd-ea66-4c2c-9345-a770a31543b3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:08.884302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7ebcd-ea66-4c2c-9345-a770a31543b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:08.884719 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7ebcd-ea66-4c2c-9345-a770a31543b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:08.893933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7ebcd-ea66-4c2c-9345-a770a31543b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:08.895975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7ebcd-ea66-4c2c-9345-a770a31543b3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-97c7ebcd-ea66-4c2c-9345-a770a31543b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:08.910617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:08.911824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:08.913428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:08.913428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:08.913428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:08.913428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:08.913428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:08.913428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:08.913428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:08.913428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:08.916167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 306/596] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:08.917248 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7ebcd-ea66-4c2c-9345-a770a31543b3 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:08.989879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:09.021185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68476e50-c6c5-48fa-8cd9-9f6bb84f48ab None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:09.048017 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7ebcd-ea66-4c2c-9345-a770a31543b3 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:09.073553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9e2b6-4511-4140-8ac8-4634577092d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:09.074204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e2b6-4511-4140-8ac8-4634577092d8 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:09.074477 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e2b6-4511-4140-8ac8-4634577092d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:09.074866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e2b6-4511-4140-8ac8-4634577092d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:09.083456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9e2b6-4511-4140-8ac8-4634577092d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:09.085873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9e2b6-4511-4140-8ac8-4634577092d8 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:09.098144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca163f-33d0-4138-b3ac-f9f14e16b1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:09.099354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca163f-33d0-4138-b3ac-f9f14e16b1c3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:09.100012 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca163f-33d0-4138-b3ac-f9f14e16b1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:09.101376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca163f-33d0-4138-b3ac-f9f14e16b1c3 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-4da9e2b6-4511-4140-8ac8-4634577092d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:09.101557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:09.102470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:09.103753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:09.103753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:09.103753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:09.103753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:09.103753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:09.103753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:09.103753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:09.103753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:09.105376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca163f-33d0-4138-b3ac-f9f14e16b1c3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:09.105913 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca163f-33d0-4138-b3ac-f9f14e16b1c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:09.107509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 291/597] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:09.108185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca163f-33d0-4138-b3ac-f9f14e16b1c3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:09.108412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9e2b6-4511-4140-8ac8-4634577092d8 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:09.131730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 307/598] 10.222.0.86 () {56 vars in 1139 bytes} [Fri Aug 5 19:37:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:09.133009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca163f-33d0-4138-b3ac-f9f14e16b1c3 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:09.170805 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:09.240222 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9e2b6-4511-4140-8ac8-4634577092d8 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:09.261461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca163f-33d0-4138-b3ac-f9f14e16b1c3 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:09.280217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa8189-8942-4721-804f-ce27d3ed3db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:09.280835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa8189-8942-4721-804f-ce27d3ed3db9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:09.281122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa8189-8942-4721-804f-ce27d3ed3db9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:09.281397 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa8189-8942-4721-804f-ce27d3ed3db9 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:09.282111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:09.282221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa8189-8942-4721-804f-ce27d3ed3db9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:09.282447 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa8189-8942-4721-804f-ce27d3ed3db9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:09.283916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa8189-8942-4721-804f-ce27d3ed3db9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:09.302700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 292/599] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:09.302700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa8189-8942-4721-804f-ce27d3ed3db9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:09.308763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89053c99-6c8a-4516-b0a7-faeb7af6d2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:09.309416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89053c99-6c8a-4516-b0a7-faeb7af6d2f5 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:09.309695 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89053c99-6c8a-4516-b0a7-faeb7af6d2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:09.309965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89053c99-6c8a-4516-b0a7-faeb7af6d2f5 None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:09.310754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89053c99-6c8a-4516-b0a7-faeb7af6d2f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:09.311061 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89053c99-6c8a-4516-b0a7-faeb7af6d2f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:09.312520 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89053c99-6c8a-4516-b0a7-faeb7af6d2f5 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:09.335235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 308/600] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:09.336720 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccf7e2-435f-4d32-87a9-6b969a20bfc0 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:09.337000 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccf7e2-435f-4d32-87a9-6b969a20bfc0 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:09.337282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccf7e2-435f-4d32-87a9-6b969a20bfc0 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:09.338920 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 309/601] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:09.339641 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccf7e2-435f-4d32-87a9-6b969a20bfc0 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:09.425757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa8189-8942-4721-804f-ce27d3ed3db9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:09.428632 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:09.471741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccf7e2-435f-4d32-87a9-6b969a20bfc0 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:09.472080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4347de-bd72-4888-bdf5-3908189c901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:09.472725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4347de-bd72-4888-bdf5-3908189c901a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:09.472995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4347de-bd72-4888-bdf5-3908189c901a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:09.473270 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4347de-bd72-4888-bdf5-3908189c901a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:09.485272 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4347de-bd72-4888-bdf5-3908189c901a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:09.486774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4347de-bd72-4888-bdf5-3908189c901a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-0a4347de-bd72-4888-bdf5-3908189c901a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:09.505455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:09.506568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:09.508071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:09.508071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:09.508071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:09.508071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:09.509182 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 293/602] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:09.509987 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4347de-bd72-4888-bdf5-3908189c901a None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:09.526201 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace49b6-2433-4f92-95c4-fee6cc44d792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:09.526968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace49b6-2433-4f92-95c4-fee6cc44d792 None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:09.527251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace49b6-2433-4f92-95c4-fee6cc44d792 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:09.527526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace49b6-2433-4f92-95c4-fee6cc44d792 None admin] PATH_INFO: `/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/afea7d279292424490beff0481ff1512/roles/02001b18565340ce984343c5d36daaf9` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:09.546096 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace49b6-2433-4f92-95c4-fee6cc44d792 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c73d2cfaa1c47829ca6936186d2bb26, user_id=afea7d279292424490beff0481ff1512, role_id=02001b18565340ce984343c5d36daaf9)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:09.547753 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace49b6-2433-4f92-95c4-fee6cc44d792 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:09.564145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 310/603] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:37:09 2022] PUT /identity/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/afea7d279292424490beff0481ff1512/roles/02001b18565340ce984343c5d36daaf9 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:37:09.566491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb88ef5-8dfa-45cf-ae03-107328be05f3 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:09.566905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb88ef5-8dfa-45cf-ae03-107328be05f3 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:09.567280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb88ef5-8dfa-45cf-ae03-107328be05f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:09.596308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-1fb88ef5-8dfa-45cf-ae03-107328be05f3 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:09.598037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1fb88ef5-8dfa-45cf-ae03-107328be05f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:09.641672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:09.673386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4347de-bd72-4888-bdf5-3908189c901a None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:09.719993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 311/604] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:09.721251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb88ef5-8dfa-45cf-ae03-107328be05f3 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:09.732317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad317d-f3bb-4152-bfc7-7047de4056d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:09.733005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad317d-f3bb-4152-bfc7-7047de4056d4 None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:09.733285 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad317d-f3bb-4152-bfc7-7047de4056d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:09.733569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad317d-f3bb-4152-bfc7-7047de4056d4 None admin] PATH_INFO: `/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/72d4459164844e718544d7fde333db76/roles/02001b18565340ce984343c5d36daaf9` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:09.758765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad317d-f3bb-4152-bfc7-7047de4056d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c73d2cfaa1c47829ca6936186d2bb26, user_id=72d4459164844e718544d7fde333db76, role_id=02001b18565340ce984343c5d36daaf9)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:09.760820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad317d-f3bb-4152-bfc7-7047de4056d4 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:09.782262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 294/605] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:37:09 2022] PUT /identity/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/72d4459164844e718544d7fde333db76/roles/02001b18565340ce984343c5d36daaf9 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:37:09.783348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad317d-f3bb-4152-bfc7-7047de4056d4 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:09.794578 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:09.894094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb88ef5-8dfa-45cf-ae03-107328be05f3 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:09.979654 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad317d-f3bb-4152-bfc7-7047de4056d4 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:09.986300 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344a0c9-1bf5-4ce0-b5fc-065b48a98437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:09.987555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a0c9-1bf5-4ce0-b5fc-065b48a98437 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:09.987835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a0c9-1bf5-4ce0-b5fc-065b48a98437 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:09.988106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344a0c9-1bf5-4ce0-b5fc-065b48a98437 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:09.989024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344a0c9-1bf5-4ce0-b5fc-065b48a98437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:09.989554 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344a0c9-1bf5-4ce0-b5fc-065b48a98437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:09.993751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344a0c9-1bf5-4ce0-b5fc-065b48a98437 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:10.017960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 312/606] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:10.023866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344a0c9-1bf5-4ce0-b5fc-065b48a98437 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:10.048348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad2f2f-de4e-49e7-9832-cc5b5e425f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:10.049013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad2f2f-de4e-49e7-9832-cc5b5e425f86 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:10.049294 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad2f2f-de4e-49e7-9832-cc5b5e425f86 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:10.049567 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad2f2f-de4e-49e7-9832-cc5b5e425f86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:10.050690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:10.064892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad2f2f-de4e-49e7-9832-cc5b5e425f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-24ad2f2f-de4e-49e7-9832-cc5b5e425f86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:10.071228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:10.072528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:10.073393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:10.073393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:10.073393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:10.073393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:10.073393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:10.073393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:10.073393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:10.073393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:10.073393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:10.073393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:10.073393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:10.073393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:10.076081 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 295/607] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:10.083129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad2f2f-de4e-49e7-9832-cc5b5e425f86 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:10.180517 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344a0c9-1bf5-4ce0-b5fc-065b48a98437 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:10.192402 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:10.253931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad2f2f-de4e-49e7-9832-cc5b5e425f86 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:10.281251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0be88-b83e-43af-b319-842fec7a7688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:10.282295 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0be88-b83e-43af-b319-842fec7a7688 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:10.282806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0be88-b83e-43af-b319-842fec7a7688 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:10.283274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e0be88-b83e-43af-b319-842fec7a7688 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:10.311024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0be88-b83e-43af-b319-842fec7a7688 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:10.314145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e0be88-b83e-43af-b319-842fec7a7688 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:10.322558 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9c4b5-bb8c-4682-9f87-9e1256949252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:10.324112 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9c4b5-bb8c-4682-9f87-9e1256949252 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:10.324841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9c4b5-bb8c-4682-9f87-9e1256949252 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:10.325569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9c4b5-bb8c-4682-9f87-9e1256949252 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:10.327123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9c4b5-bb8c-4682-9f87-9e1256949252 None admin] RBAC: Adding query filter params (name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:10.327919 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9c4b5-bb8c-4682-9f87-9e1256949252 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:10.330648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9c4b5-bb8c-4682-9f87-9e1256949252 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-24e0be88-b83e-43af-b319-842fec7a7688 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:10.349141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:10.352043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:10.359965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:10.359965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:10.359965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:10.359965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:10.359965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:10.359965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:10.359965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:10.359965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:10.361547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 313/608] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:10.362114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 296/609] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:10.368776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0be88-b83e-43af-b319-842fec7a7688 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:10.372468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9c4b5-bb8c-4682-9f87-9e1256949252 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:10.499669 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9c4b5-bb8c-4682-9f87-9e1256949252 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:10.511951 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:10.536417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:10.546657 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4b295-64f1-4617-9881-b41e7e167923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:10.547637 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4b295-64f1-4617-9881-b41e7e167923 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:10.548138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4b295-64f1-4617-9881-b41e7e167923 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:10.548253 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e0be88-b83e-43af-b319-842fec7a7688 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:10.548715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4b295-64f1-4617-9881-b41e7e167923 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:10.549666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4b295-64f1-4617-9881-b41e7e167923 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:10.550174 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4b295-64f1-4617-9881-b41e7e167923 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:10.551841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4b295-64f1-4617-9881-b41e7e167923 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:10.566360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 297/610] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:10.568083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d8b9b-585b-4360-8c8c-2400b490898a None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:10.568566 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d8b9b-585b-4360-8c8c-2400b490898a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:10.569998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d8b9b-585b-4360-8c8c-2400b490898a None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:10.573226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 298/611] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:10.574352 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d8b9b-585b-4360-8c8c-2400b490898a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:10.597475 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a25da-9aaf-4e0b-b992-e864fe0ddb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:10.598488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a25da-9aaf-4e0b-b992-e864fe0ddb64 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:10.598993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a25da-9aaf-4e0b-b992-e864fe0ddb64 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:10.599473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a25da-9aaf-4e0b-b992-e864fe0ddb64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:10.612728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a25da-9aaf-4e0b-b992-e864fe0ddb64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:10.614526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a25da-9aaf-4e0b-b992-e864fe0ddb64 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-4f2a25da-9aaf-4e0b-b992-e864fe0ddb64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:10.632238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:10.633644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:10.635013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:10.635013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:10.635013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:10.635013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:10.642181 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 314/612] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:10.644049 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d201096-63fb-43f4-ace0-9ba58d534c1e None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:10.644539 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d201096-63fb-43f4-ace0-9ba58d534c1e None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:10.645003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d201096-63fb-43f4-ace0-9ba58d534c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:10.665158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-5d201096-63fb-43f4-ace0-9ba58d534c1e None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:10.667063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5d201096-63fb-43f4-ace0-9ba58d534c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:10.729311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d8b9b-585b-4360-8c8c-2400b490898a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:10.745949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 315/613] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:10.747329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d201096-63fb-43f4-ace0-9ba58d534c1e None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:10.799612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b0d18-81f0-43c1-a8fa-0dbe068df88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:10.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b0d18-81f0-43c1-a8fa-0dbe068df88b None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:10.800538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b0d18-81f0-43c1-a8fa-0dbe068df88b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:10.800859 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b0d18-81f0-43c1-a8fa-0dbe068df88b None admin] PATH_INFO: `/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/80097afc3ce540e6a8060f696535e62c/roles/02001b18565340ce984343c5d36daaf9` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:10.815457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:10.821099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b0d18-81f0-43c1-a8fa-0dbe068df88b None admin] RBAC: Authorizing `identity:create_grant(project_id=9c73d2cfaa1c47829ca6936186d2bb26, user_id=80097afc3ce540e6a8060f696535e62c, role_id=02001b18565340ce984343c5d36daaf9)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:10.822582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b0d18-81f0-43c1-a8fa-0dbe068df88b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:10.840575 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 299/614] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:37:10 2022] PUT /identity/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/80097afc3ce540e6a8060f696535e62c/roles/02001b18565340ce984343c5d36daaf9 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:37:10.841498 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b0d18-81f0-43c1-a8fa-0dbe068df88b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:10.873254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d201096-63fb-43f4-ace0-9ba58d534c1e None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:10.900543 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:10.922010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d805885-8db0-49bf-9929-8d3da8e2ab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:10.923059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d805885-8db0-49bf-9929-8d3da8e2ab0f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:10.923603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d805885-8db0-49bf-9929-8d3da8e2ab0f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:10.924021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d805885-8db0-49bf-9929-8d3da8e2ab0f None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:10.924934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d805885-8db0-49bf-9929-8d3da8e2ab0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:10.925448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d805885-8db0-49bf-9929-8d3da8e2ab0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:10.927138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d805885-8db0-49bf-9929-8d3da8e2ab0f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:10.942255 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 316/615] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:10.952129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d805885-8db0-49bf-9929-8d3da8e2ab0f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:11.044414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b0d18-81f0-43c1-a8fa-0dbe068df88b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:11.076012 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d805885-8db0-49bf-9929-8d3da8e2ab0f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:11.113386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bf9fb-9943-4aae-bdbc-ea3545aac2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:11.114477 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf9fb-9943-4aae-bdbc-ea3545aac2ca None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:11.114999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf9fb-9943-4aae-bdbc-ea3545aac2ca None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:11.115915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bf9fb-9943-4aae-bdbc-ea3545aac2ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:11.117196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:11.133726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bf9fb-9943-4aae-bdbc-ea3545aac2ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-757bf9fb-9943-4aae-bdbc-ea3545aac2ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:11.139284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:11.140263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:11.141093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:11.141093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:11.141093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:11.141093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:11.141093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:11.141093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:11.141093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:11.141093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:11.141093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:11.141093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:11.141093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:11.141093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:11.141715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 300/616] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:11.148380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bf9fb-9943-4aae-bdbc-ea3545aac2ca None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:11.168569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa9f22-119f-4c5d-a9af-99dd3c40fe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:11.169702 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa9f22-119f-4c5d-a9af-99dd3c40fe98 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:11.170126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa9f22-119f-4c5d-a9af-99dd3c40fe98 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:11.170759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa9f22-119f-4c5d-a9af-99dd3c40fe98 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:11.182073 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa9f22-119f-4c5d-a9af-99dd3c40fe98 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:11.183713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa9f22-119f-4c5d-a9af-99dd3c40fe98 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-6bfa9f22-119f-4c5d-a9af-99dd3c40fe98 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:11.200058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:11.201069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:11.202007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:11.202007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:11.202007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:11.202007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:11.202007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:11.202007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:11.202007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:11.202007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:11.202007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:37:11.202007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:11.203938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 317/617] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:37:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:11.210519 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa9f22-119f-4c5d-a9af-99dd3c40fe98 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:11.328582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bf9fb-9943-4aae-bdbc-ea3545aac2ca None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:11.330766 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:11.398775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa9f22-119f-4c5d-a9af-99dd3c40fe98 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:11.423458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7b559-594a-47bf-b208-0ce4a9843925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:11.424105 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7b559-594a-47bf-b208-0ce4a9843925 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:11.424388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7b559-594a-47bf-b208-0ce4a9843925 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:11.424666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7b559-594a-47bf-b208-0ce4a9843925 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:11.425672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7b559-594a-47bf-b208-0ce4a9843925 None admin] RBAC: Adding query filter params (name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:11.426209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7b559-594a-47bf-b208-0ce4a9843925 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:11.429311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7b559-594a-47bf-b208-0ce4a9843925 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:11.440475 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18338414-9954-448c-be4c-9001c3c475c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:11.441246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18338414-9954-448c-be4c-9001c3c475c1 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:11.441637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18338414-9954-448c-be4c-9001c3c475c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:11.442013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18338414-9954-448c-be4c-9001c3c475c1 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:11.442993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18338414-9954-448c-be4c-9001c3c475c1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:11.443638 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18338414-9954-448c-be4c-9001c3c475c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:11.446436 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18338414-9954-448c-be4c-9001c3c475c1 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:11.451150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 301/618] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:11.467710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7b559-594a-47bf-b208-0ce4a9843925 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:11.472593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 318/619] 10.222.0.86 () {56 vars in 1135 bytes} [Fri Aug 5 19:37:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:11.482949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18338414-9954-448c-be4c-9001c3c475c1 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:11.577781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:11.655075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7b559-594a-47bf-b208-0ce4a9843925 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:11.731192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fca79-8a45-4b19-9c4f-0de0c75d9724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:11.731874 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fca79-8a45-4b19-9c4f-0de0c75d9724 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:11.732326 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fca79-8a45-4b19-9c4f-0de0c75d9724 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:11.732796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fca79-8a45-4b19-9c4f-0de0c75d9724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:11.739187 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18338414-9954-448c-be4c-9001c3c475c1 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:11.755837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4fca79-8a45-4b19-9c4f-0de0c75d9724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:11.758309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4fca79-8a45-4b19-9c4f-0de0c75d9724 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-4e4fca79-8a45-4b19-9c4f-0de0c75d9724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:11.789066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:11.790162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:11.791112 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:11.791112 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:11.791112 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:11.791112 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:11.797414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 302/620] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:11.814130 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fca79-8a45-4b19-9c4f-0de0c75d9724 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:11.829707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bdeaa-a753-4655-a2e5-3bcdd1cef1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:11.830360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bdeaa-a753-4655-a2e5-3bcdd1cef1f6 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:11.830709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bdeaa-a753-4655-a2e5-3bcdd1cef1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:11.830980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bdeaa-a753-4655-a2e5-3bcdd1cef1f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:11.838997 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:11.854717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11bdeaa-a753-4655-a2e5-3bcdd1cef1f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:11.856216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11bdeaa-a753-4655-a2e5-3bcdd1cef1f6 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-d11bdeaa-a753-4655-a2e5-3bcdd1cef1f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:11.885052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:11.886028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:11.887044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:11.887044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:11.887044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:11.887044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:11.893876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 319/621] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:11.895148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bdeaa-a753-4655-a2e5-3bcdd1cef1f6 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:12.015631 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fca79-8a45-4b19-9c4f-0de0c75d9724 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:12.050940 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:12.073117 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bdeaa-a753-4655-a2e5-3bcdd1cef1f6 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:12.085901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e5f85-3bc0-44b9-bdfc-afea9bad62e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:12.087084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e5f85-3bc0-44b9-bdfc-afea9bad62e6 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:12.087625 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e5f85-3bc0-44b9-bdfc-afea9bad62e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:12.088097 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e5f85-3bc0-44b9-bdfc-afea9bad62e6 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:12.088997 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e5f85-3bc0-44b9-bdfc-afea9bad62e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:12.089521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e5f85-3bc0-44b9-bdfc-afea9bad62e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:12.091142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e5f85-3bc0-44b9-bdfc-afea9bad62e6 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:12.118048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 303/622] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:12.122879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e5f85-3bc0-44b9-bdfc-afea9bad62e6 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:12.149730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63439c2-561d-47ec-a310-e78f18e14b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:12.151474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63439c2-561d-47ec-a310-e78f18e14b12 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:12.152246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63439c2-561d-47ec-a310-e78f18e14b12 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:12.152990 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63439c2-561d-47ec-a310-e78f18e14b12 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:12.154544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63439c2-561d-47ec-a310-e78f18e14b12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:12.155434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63439c2-561d-47ec-a310-e78f18e14b12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:12.157811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63439c2-561d-47ec-a310-e78f18e14b12 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:12.180614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 320/623] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:12.191712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85972124-b168-4277-9067-b76d9d8a1981 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:12.192017 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85972124-b168-4277-9067-b76d9d8a1981 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:12.192270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85972124-b168-4277-9067-b76d9d8a1981 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:12.198102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 321/624] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:37:12.199056 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85972124-b168-4277-9067-b76d9d8a1981 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:12.247193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:12.282198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e5f85-3bc0-44b9-bdfc-afea9bad62e6 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:12.318383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85972124-b168-4277-9067-b76d9d8a1981 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:12.320560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3151e4f0-d32a-4c76-85c6-cb6bb64f36af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:12.321193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151e4f0-d32a-4c76-85c6-cb6bb64f36af None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:12.321466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151e4f0-d32a-4c76-85c6-cb6bb64f36af None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:12.321734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3151e4f0-d32a-4c76-85c6-cb6bb64f36af None admin] PATH_INFO: `/v3/users/nova` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:12.329403 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151e4f0-d32a-4c76-85c6-cb6bb64f36af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:12.330945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3151e4f0-d32a-4c76-85c6-cb6bb64f36af None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-3151e4f0-d32a-4c76-85c6-cb6bb64f36af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:12.344219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:12.345247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:12.346200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:12.346200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:12.346200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:12.346200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:12.346200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:12.346200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:12.346200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:12.346200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:12.346200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:37:12.346200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:12.346200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 304/625] 10.222.0.86 () {56 vars in 1093 bytes} [Fri Aug 5 19:37:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:12.346819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c8baa-e07f-44df-9060-72114412a0f2 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:12.346819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c8baa-e07f-44df-9060-72114412a0f2 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:12.347540 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c8baa-e07f-44df-9060-72114412a0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:12.365810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-d93c8baa-e07f-44df-9060-72114412a0f2 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:12.367079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d93c8baa-e07f-44df-9060-72114412a0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:12.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16bb005-7c89-4865-b4a6-dea32d6832ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:12.371019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bb005-7c89-4865-b4a6-dea32d6832ec None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:12.371304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bb005-7c89-4865-b4a6-dea32d6832ec None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:12.371826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16bb005-7c89-4865-b4a6-dea32d6832ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:12.379656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16bb005-7c89-4865-b4a6-dea32d6832ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:12.381119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16bb005-7c89-4865-b4a6-dea32d6832ec None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-b16bb005-7c89-4865-b4a6-dea32d6832ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:12.394381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:12.395468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:12.396361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:12.396361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:12.396361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:12.396361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:12.396361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:12.396361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:12.396361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:12.396361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:12.396361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 322/626] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:12.397548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506d883-db54-48f1-81c6-181437457bb7 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:12.398682 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506d883-db54-48f1-81c6-181437457bb7 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:12.398682 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506d883-db54-48f1-81c6-181437457bb7 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:12.404412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 323/627] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:12.404412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506d883-db54-48f1-81c6-181437457bb7 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:12.417629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:12.462935 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 305/628] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:12.469827 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c8baa-e07f-44df-9060-72114412a0f2 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:12.543096 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506d883-db54-48f1-81c6-181437457bb7 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:12.548013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:12.582113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c03ad-e553-46a1-a147-fa2d3e583020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:12.582794 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c03ad-e553-46a1-a147-fa2d3e583020 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:12.583075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c03ad-e553-46a1-a147-fa2d3e583020 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:12.583346 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c03ad-e553-46a1-a147-fa2d3e583020 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:12.584142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c03ad-e553-46a1-a147-fa2d3e583020 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:12.584989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c03ad-e553-46a1-a147-fa2d3e583020 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:12.586453 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c03ad-e553-46a1-a147-fa2d3e583020 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:12.599774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c8baa-e07f-44df-9060-72114412a0f2 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:12.612010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 324/629] 10.222.0.86 () {56 vars in 1133 bytes} [Fri Aug 5 19:37:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:12.612010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c74ff7-02c9-4fd3-83f6-2f853cf1d978 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:12.612010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c74ff7-02c9-4fd3-83f6-2f853cf1d978 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:12.612010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c74ff7-02c9-4fd3-83f6-2f853cf1d978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:12.627972 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-94c74ff7-02c9-4fd3-83f6-2f853cf1d978 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:12.629334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-94c74ff7-02c9-4fd3-83f6-2f853cf1d978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:12.643214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0730cb6-f528-4e35-a953-cdf58c5aca3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:12.643817 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0730cb6-f528-4e35-a953-cdf58c5aca3f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:12.644140 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0730cb6-f528-4e35-a953-cdf58c5aca3f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:12.644349 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0730cb6-f528-4e35-a953-cdf58c5aca3f None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:12.645193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0730cb6-f528-4e35-a953-cdf58c5aca3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:12.645390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0730cb6-f528-4e35-a953-cdf58c5aca3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:12.646811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0730cb6-f528-4e35-a953-cdf58c5aca3f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:12.659658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 306/630] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:12.660364 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0730cb6-f528-4e35-a953-cdf58c5aca3f None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:12.702672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:12.712622 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 325/631] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:12.714318 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c74ff7-02c9-4fd3-83f6-2f853cf1d978 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:12.782445 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0730cb6-f528-4e35-a953-cdf58c5aca3f None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:12.809302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:12.823688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a11f5d-3bdd-4669-bbab-c29e3ec66b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:12.824325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a11f5d-3bdd-4669-bbab-c29e3ec66b25 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:12.824602 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a11f5d-3bdd-4669-bbab-c29e3ec66b25 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:12.824872 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a11f5d-3bdd-4669-bbab-c29e3ec66b25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:12.836951 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a11f5d-3bdd-4669-bbab-c29e3ec66b25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-44a11f5d-3bdd-4669-bbab-c29e3ec66b25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:12.843662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:12.844672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 307/632] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:12.845512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a11f5d-3bdd-4669-bbab-c29e3ec66b25 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:12.866358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c74ff7-02c9-4fd3-83f6-2f853cf1d978 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:12.905537 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4b51e-cd6c-4a64-b6ba-5a0498889cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:12.906516 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4b51e-cd6c-4a64-b6ba-5a0498889cca None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:12.906959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4b51e-cd6c-4a64-b6ba-5a0498889cca None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:12.907363 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4b51e-cd6c-4a64-b6ba-5a0498889cca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:12.979532 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:12.981792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a11f5d-3bdd-4669-bbab-c29e3ec66b25 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:12.988493 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4b51e-cd6c-4a64-b6ba-5a0498889cca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:12.991935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4b51e-cd6c-4a64-b6ba-5a0498889cca None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:13.019569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cca8c1-78ab-44d7-9fe6-97881f1e029b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:13.020237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cca8c1-78ab-44d7-9fe6-97881f1e029b None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:13.020528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cca8c1-78ab-44d7-9fe6-97881f1e029b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:13.020792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cca8c1-78ab-44d7-9fe6-97881f1e029b None admin] PATH_INFO: `/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/e8adb0fd1fbc4268b70f3936036e550b/roles/02001b18565340ce984343c5d36daaf9` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:13.037192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cca8c1-78ab-44d7-9fe6-97881f1e029b None admin] RBAC: Authorizing `identity:create_grant(project_id=9c73d2cfaa1c47829ca6936186d2bb26, user_id=e8adb0fd1fbc4268b70f3936036e550b, role_id=02001b18565340ce984343c5d36daaf9)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:13.038716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cca8c1-78ab-44d7-9fe6-97881f1e029b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:13.054441 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 308/633] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:37:12 2022] PUT /identity/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/e8adb0fd1fbc4268b70f3936036e550b/roles/02001b18565340ce984343c5d36daaf9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:37:13.055305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cca8c1-78ab-44d7-9fe6-97881f1e029b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:13.109932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:13.158286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 326/634] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:37:13.158286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4b51e-cd6c-4a64-b6ba-5a0498889cca None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:13.224860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cca8c1-78ab-44d7-9fe6-97881f1e029b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:13.273997 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f4b27-572b-49ed-b9b0-360fa46559d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:13.274667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f4b27-572b-49ed-b9b0-360fa46559d1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:13.274995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f4b27-572b-49ed-b9b0-360fa46559d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:13.275312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f4b27-572b-49ed-b9b0-360fa46559d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:13.291255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f4b27-572b-49ed-b9b0-360fa46559d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:13.298460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-6d4f4b27-572b-49ed-b9b0-360fa46559d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:13.300765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:13.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:13.302538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:13.302538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:13.302538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:13.302538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:13.302538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:13.302538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:13.302538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:13.302538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:13.302538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:13.302538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:13.302538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:13.302538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:13.303815 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 309/635] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:13.305873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f4b27-572b-49ed-b9b0-360fa46559d1 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:13.307754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4b51e-cd6c-4a64-b6ba-5a0498889cca None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:13.360608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889ba9b-45d5-45fd-aa41-154b39b6fb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:13.361145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889ba9b-45d5-45fd-aa41-154b39b6fb89 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:13.361419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889ba9b-45d5-45fd-aa41-154b39b6fb89 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:13.361688 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889ba9b-45d5-45fd-aa41-154b39b6fb89 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:13.362409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889ba9b-45d5-45fd-aa41-154b39b6fb89 None admin] RBAC: Adding query filter params (name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:13.362734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889ba9b-45d5-45fd-aa41-154b39b6fb89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:13.364121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889ba9b-45d5-45fd-aa41-154b39b6fb89 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:13.374146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 327/636] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:13.375030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889ba9b-45d5-45fd-aa41-154b39b6fb89 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:13.428888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:13.440495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f4b27-572b-49ed-b9b0-360fa46559d1 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:13.481314 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ed17f-a4e9-434e-9749-8450bb3444a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:13.481802 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ed17f-a4e9-434e-9749-8450bb3444a5 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:13.482085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ed17f-a4e9-434e-9749-8450bb3444a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:13.482355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ed17f-a4e9-434e-9749-8450bb3444a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:13.493429 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889ba9b-45d5-45fd-aa41-154b39b6fb89 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:13.494803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ed17f-a4e9-434e-9749-8450bb3444a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:13.496426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ed17f-a4e9-434e-9749-8450bb3444a5 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-536ed17f-a4e9-434e-9749-8450bb3444a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:13.521230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:13.523211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:13.524098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:13.524098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:13.524098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:13.524098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:13.524098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 310/637] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:13.524879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae24e1-267a-469b-a3fa-91105af898ae None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:13.525314 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae24e1-267a-469b-a3fa-91105af898ae None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:13.525472 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae24e1-267a-469b-a3fa-91105af898ae None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:13.530775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 311/638] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:13.530775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae24e1-267a-469b-a3fa-91105af898ae None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:13.545384 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7ae15-ef15-4731-a234-5f250cfed2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:13.546006 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7ae15-ef15-4731-a234-5f250cfed2af None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:13.546283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7ae15-ef15-4731-a234-5f250cfed2af None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:13.546548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7ae15-ef15-4731-a234-5f250cfed2af None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:13.547292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7ae15-ef15-4731-a234-5f250cfed2af None admin] RBAC: Adding query filter params (name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:13.547585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7ae15-ef15-4731-a234-5f250cfed2af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:13.549085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7ae15-ef15-4731-a234-5f250cfed2af None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:13.564125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 328/639] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:13.564125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7ae15-ef15-4731-a234-5f250cfed2af None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:13.590365 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:13.655957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae24e1-267a-469b-a3fa-91105af898ae None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:13.685755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7ae15-ef15-4731-a234-5f250cfed2af None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:13.694125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de683a5-026d-4b80-8868-c950c50af34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:13.695095 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de683a5-026d-4b80-8868-c950c50af34f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:13.695531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de683a5-026d-4b80-8868-c950c50af34f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:13.695924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de683a5-026d-4b80-8868-c950c50af34f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:13.707419 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de683a5-026d-4b80-8868-c950c50af34f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:13.709087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de683a5-026d-4b80-8868-c950c50af34f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:13.713639 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:13.727628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d767899-664e-4a96-9b6a-34afaa707a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:13.728386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d767899-664e-4a96-9b6a-34afaa707a0c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:13.728767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d767899-664e-4a96-9b6a-34afaa707a0c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:13.729157 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d767899-664e-4a96-9b6a-34afaa707a0c None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:13.729954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d767899-664e-4a96-9b6a-34afaa707a0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:13.730357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d767899-664e-4a96-9b6a-34afaa707a0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-3de683a5-026d-4b80-8868-c950c50af34f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:13.732737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:13.733697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:13.734699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:13.734699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:13.734699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:13.734699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:13.734923 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d767899-664e-4a96-9b6a-34afaa707a0c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:13.735011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 312/640] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:13.735011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e96a10-9a39-43e4-aa21-9367c4b64492 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:13.735011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e96a10-9a39-43e4-aa21-9367c4b64492 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:13.735011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e96a10-9a39-43e4-aa21-9367c4b64492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:13.750403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 329/641] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:13.752255 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d767899-664e-4a96-9b6a-34afaa707a0c None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:13.757657 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-e9e96a10-9a39-43e4-aa21-9367c4b64492 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:13.758922 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e9e96a10-9a39-43e4-aa21-9367c4b64492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:13.844645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 313/642] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:13.845734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e96a10-9a39-43e4-aa21-9367c4b64492 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:13.869652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:13.891952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d767899-664e-4a96-9b6a-34afaa707a0c None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:13.932616 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd794a0-646c-464c-bf6a-2caf30faa028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:13.933249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd794a0-646c-464c-bf6a-2caf30faa028 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:13.933935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd794a0-646c-464c-bf6a-2caf30faa028 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:13.933935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd794a0-646c-464c-bf6a-2caf30faa028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:13.946800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd794a0-646c-464c-bf6a-2caf30faa028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:13.948548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd794a0-646c-464c-bf6a-2caf30faa028 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-3cd794a0-646c-464c-bf6a-2caf30faa028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:13.965044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:13.966108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:13.967078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:13.967078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:13.967078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:13.967078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:13.967078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 330/643] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:13.967078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd794a0-646c-464c-bf6a-2caf30faa028 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:13.976373 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e96a10-9a39-43e4-aa21-9367c4b64492 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:14.012226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:14.022972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57678589-2bc8-4ba4-b6e0-921ce3421f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:14.023737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57678589-2bc8-4ba4-b6e0-921ce3421f29 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:14.024098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57678589-2bc8-4ba4-b6e0-921ce3421f29 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:14.024362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57678589-2bc8-4ba4-b6e0-921ce3421f29 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:14.025157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57678589-2bc8-4ba4-b6e0-921ce3421f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:14.025465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57678589-2bc8-4ba4-b6e0-921ce3421f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:14.027014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57678589-2bc8-4ba4-b6e0-921ce3421f29 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:14.040744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 314/644] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:14.041610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57678589-2bc8-4ba4-b6e0-921ce3421f29 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:14.099312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd794a0-646c-464c-bf6a-2caf30faa028 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:14.139009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:14.147281 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac461fb6-3def-45ab-898c-bb4bb7854965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:14.148270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac461fb6-3def-45ab-898c-bb4bb7854965 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:14.148740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac461fb6-3def-45ab-898c-bb4bb7854965 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:14.149229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac461fb6-3def-45ab-898c-bb4bb7854965 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:14.160347 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac461fb6-3def-45ab-898c-bb4bb7854965 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:14.161809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac461fb6-3def-45ab-898c-bb4bb7854965 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:14.172885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57678589-2bc8-4ba4-b6e0-921ce3421f29 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-ac461fb6-3def-45ab-898c-bb4bb7854965 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:14.175367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:14.176430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:14.177634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:14.177634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:14.177634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:14.177634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:14.177634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:14.177634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:14.177634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:14.177634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:14.177634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 331/645] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:14.183019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac461fb6-3def-45ab-898c-bb4bb7854965 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:14.213872 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9bad2-7fa5-46c1-a9a7-a82303bc6d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:14.214518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9bad2-7fa5-46c1-a9a7-a82303bc6d92 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:14.214834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9bad2-7fa5-46c1-a9a7-a82303bc6d92 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:14.215109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9bad2-7fa5-46c1-a9a7-a82303bc6d92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:14.226088 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9bad2-7fa5-46c1-a9a7-a82303bc6d92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-4ea9bad2-7fa5-46c1-a9a7-a82303bc6d92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:14.231700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:14.232858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:14.233883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:14.233883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:14.233883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:14.233883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:14.233883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:14.233883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:14.233883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:14.233883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:14.233883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:14.233883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:14.233883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:14.233883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:14.237753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 315/646] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:14.238694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9bad2-7fa5-46c1-a9a7-a82303bc6d92 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:14.297898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:14.311038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac461fb6-3def-45ab-898c-bb4bb7854965 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:14.353846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bda3a-6207-4f69-b639-a5e6e2e9498d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:14.354852 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bda3a-6207-4f69-b639-a5e6e2e9498d None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:14.355317 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bda3a-6207-4f69-b639-a5e6e2e9498d None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:14.355600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bda3a-6207-4f69-b639-a5e6e2e9498d None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:14.356316 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bda3a-6207-4f69-b639-a5e6e2e9498d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:14.356624 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bda3a-6207-4f69-b639-a5e6e2e9498d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:14.358023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bda3a-6207-4f69-b639-a5e6e2e9498d None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:14.368040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9bad2-7fa5-46c1-a9a7-a82303bc6d92 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:14.371609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 332/647] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:14.372006 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bda3a-6207-4f69-b639-a5e6e2e9498d None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:14.410425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72587d7-969e-4460-ba79-33a9f301d1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:14.411272 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72587d7-969e-4460-ba79-33a9f301d1e9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:14.411550 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72587d7-969e-4460-ba79-33a9f301d1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:14.412124 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72587d7-969e-4460-ba79-33a9f301d1e9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:14.422230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72587d7-969e-4460-ba79-33a9f301d1e9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:14.424601 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72587d7-969e-4460-ba79-33a9f301d1e9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-b72587d7-969e-4460-ba79-33a9f301d1e9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:14.442235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:14.444307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:14.445252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:14.445252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:14.445252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:14.445252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:14.445252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:14.445252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:14.445252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:14.445252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:14.445252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:37:14.445252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:14.445252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 316/648] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:37:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:14.446927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72587d7-969e-4460-ba79-33a9f301d1e9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:14.454368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:14.520390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bda3a-6207-4f69-b639-a5e6e2e9498d None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:14.563491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e8992-53d1-4a27-bab3-7ff769447e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:14.564101 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e8992-53d1-4a27-bab3-7ff769447e0b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:14.564377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e8992-53d1-4a27-bab3-7ff769447e0b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:14.564645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e8992-53d1-4a27-bab3-7ff769447e0b None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:14.565393 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e8992-53d1-4a27-bab3-7ff769447e0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:14.565692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e8992-53d1-4a27-bab3-7ff769447e0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:14.567151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e8992-53d1-4a27-bab3-7ff769447e0b None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:14.568160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72587d7-969e-4460-ba79-33a9f301d1e9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:14.577484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:14.580652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 333/649] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:14.586608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e8992-53d1-4a27-bab3-7ff769447e0b None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:14.612313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7ac5c-13a9-4cbb-925e-1b164bd357ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:14.613111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7ac5c-13a9-4cbb-925e-1b164bd357ec None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:14.613594 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7ac5c-13a9-4cbb-925e-1b164bd357ec None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:14.614131 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7ac5c-13a9-4cbb-925e-1b164bd357ec None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:14.615330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7ac5c-13a9-4cbb-925e-1b164bd357ec None admin] RBAC: Adding query filter params (name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:14.616748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7ac5c-13a9-4cbb-925e-1b164bd357ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:14.618743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7ac5c-13a9-4cbb-925e-1b164bd357ec None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:14.633566 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 317/650] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:14.633566 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7ac5c-13a9-4cbb-925e-1b164bd357ec None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:14.708319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e8992-53d1-4a27-bab3-7ff769447e0b None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:14.723945 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:14.748223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6ebf8-4341-4b4b-9dc3-3796e8a20ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:14.748796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6ebf8-4341-4b4b-9dc3-3796e8a20ce4 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:14.749069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6ebf8-4341-4b4b-9dc3-3796e8a20ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:14.749337 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6ebf8-4341-4b4b-9dc3-3796e8a20ce4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:14.754302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7ac5c-13a9-4cbb-925e-1b164bd357ec None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:14.760267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6ebf8-4341-4b4b-9dc3-3796e8a20ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:14.760267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6ebf8-4341-4b4b-9dc3-3796e8a20ce4 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-9de6ebf8-4341-4b4b-9dc3-3796e8a20ce4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:14.772373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:14.773685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:14.774906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:14.774906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:14.774906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:14.774906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:14.774906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:14.774906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:14.774906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:14.774906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:14.774906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:37:14.774906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:14.776048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 334/651] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:37:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:14.781141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6ebf8-4341-4b4b-9dc3-3796e8a20ce4 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:14.801082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb716fc-bee9-4e9b-90bd-bcff299463a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:14.802153 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb716fc-bee9-4e9b-90bd-bcff299463a9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:14.802495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb716fc-bee9-4e9b-90bd-bcff299463a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:14.802857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb716fc-bee9-4e9b-90bd-bcff299463a9 None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:14.803776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb716fc-bee9-4e9b-90bd-bcff299463a9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:14.804147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb716fc-bee9-4e9b-90bd-bcff299463a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:14.805647 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb716fc-bee9-4e9b-90bd-bcff299463a9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:14.825652 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 318/652] 10.222.0.86 () {56 vars in 1137 bytes} [Fri Aug 5 19:37:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:14.826589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb716fc-bee9-4e9b-90bd-bcff299463a9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:14.858784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:14.905688 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6ebf8-4341-4b4b-9dc3-3796e8a20ce4 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:14.945628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbc443-9582-4dea-8953-16d1cf5d6375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:14.946190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbc443-9582-4dea-8953-16d1cf5d6375 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:14.946526 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb716fc-bee9-4e9b-90bd-bcff299463a9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:14.946789 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbc443-9582-4dea-8953-16d1cf5d6375 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:14.946944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbc443-9582-4dea-8953-16d1cf5d6375 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:14.947801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbc443-9582-4dea-8953-16d1cf5d6375 None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=ab8f9d9cf4ca4e38ae6eb67e949b40fd) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:14.948175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbc443-9582-4dea-8953-16d1cf5d6375 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:14.950486 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbc443-9582-4dea-8953-16d1cf5d6375 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:14.967069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 335/653] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:14 2022] GET /identity/v3/role_assignments?user.id=ab8f9d9cf4ca4e38ae6eb67e949b40fd&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:14.967069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbc443-9582-4dea-8953-16d1cf5d6375 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:14.989317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a9e92-b944-45b3-81d7-dbbb38c62662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:14.990187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a9e92-b944-45b3-81d7-dbbb38c62662 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:14.990722 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a9e92-b944-45b3-81d7-dbbb38c62662 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:14.991224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a9e92-b944-45b3-81d7-dbbb38c62662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:15.005311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a9e92-b944-45b3-81d7-dbbb38c62662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:15.008610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a9e92-b944-45b3-81d7-dbbb38c62662 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-f87a9e92-b944-45b3-81d7-dbbb38c62662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:15.027111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:15.028310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:15.029418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:15.029418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:15.029418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:15.029418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:15.031071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:15.035573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 319/654] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:15.036767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a9e92-b944-45b3-81d7-dbbb38c62662 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:15.122521 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbc443-9582-4dea-8953-16d1cf5d6375 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:15.148762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:15.170948 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9b135-8f18-41d0-a489-0812bd532d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:15.171569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9b135-8f18-41d0-a489-0812bd532d66 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:15.171843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9b135-8f18-41d0-a489-0812bd532d66 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:15.172111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9b135-8f18-41d0-a489-0812bd532d66 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:15.173063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9b135-8f18-41d0-a489-0812bd532d66 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:15.173360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9b135-8f18-41d0-a489-0812bd532d66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:15.175599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9b135-8f18-41d0-a489-0812bd532d66 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:15.180971 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a9e92-b944-45b3-81d7-dbbb38c62662 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:15.201589 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 336/655] 10.222.0.86 () {56 vars in 1137 bytes} [Fri Aug 5 19:37:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:15.202467 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9b135-8f18-41d0-a489-0812bd532d66 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:15.237389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99872bdd-88b1-4307-b72c-9fe1ffd6f371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:15.238138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99872bdd-88b1-4307-b72c-9fe1ffd6f371 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:15.238425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99872bdd-88b1-4307-b72c-9fe1ffd6f371 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:15.238725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99872bdd-88b1-4307-b72c-9fe1ffd6f371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:15.321254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:15.328372 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99872bdd-88b1-4307-b72c-9fe1ffd6f371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:15.328690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9b135-8f18-41d0-a489-0812bd532d66 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:15.330407 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99872bdd-88b1-4307-b72c-9fe1ffd6f371 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:15.366288 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89495f5a-4c19-4bdc-9b37-8a8266fdb1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:15.366944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89495f5a-4c19-4bdc-9b37-8a8266fdb1ca None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:15.367228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89495f5a-4c19-4bdc-9b37-8a8266fdb1ca None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:15.367499 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89495f5a-4c19-4bdc-9b37-8a8266fdb1ca None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:15.368198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89495f5a-4c19-4bdc-9b37-8a8266fdb1ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:15.368598 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89495f5a-4c19-4bdc-9b37-8a8266fdb1ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:15.370006 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89495f5a-4c19-4bdc-9b37-8a8266fdb1ca None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:15.386610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 337/656] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:15.386610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89495f5a-4c19-4bdc-9b37-8a8266fdb1ca None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:15.426563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:15.451923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 320/657] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:37:15.453206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12cec9-2ed5-4319-8a11-7ac721c65169 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:15.453490 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12cec9-2ed5-4319-8a11-7ac721c65169 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:15.453762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12cec9-2ed5-4319-8a11-7ac721c65169 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:15.458966 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 321/658] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:15.458966 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12cec9-2ed5-4319-8a11-7ac721c65169 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:15.530891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89495f5a-4c19-4bdc-9b37-8a8266fdb1ca None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:15.574669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9b2b9-48d4-493d-853a-4e41c865468e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:15.575604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9b2b9-48d4-493d-853a-4e41c865468e None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:15.576142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9b2b9-48d4-493d-853a-4e41c865468e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:15.576611 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9b2b9-48d4-493d-853a-4e41c865468e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:15.585278 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12cec9-2ed5-4319-8a11-7ac721c65169 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:15.605399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:15.627744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99d4e9-a4e5-461e-98a5-c78da1e49b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:15.629009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99d4e9-a4e5-461e-98a5-c78da1e49b99 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:15.629503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99d4e9-a4e5-461e-98a5-c78da1e49b99 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:15.630009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99d4e9-a4e5-461e-98a5-c78da1e49b99 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:15.639082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99d4e9-a4e5-461e-98a5-c78da1e49b99 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:15.640911 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99d4e9-a4e5-461e-98a5-c78da1e49b99 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-9e99d4e9-a4e5-461e-98a5-c78da1e49b99 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:15.653836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:15.655099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:15.656206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:15.656206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:15.656206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:15.656206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:15.656206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:15.656206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:15.656206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:15.656206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:15.656206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:37:15.656206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:15.656206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 322/659] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:15.660746 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99d4e9-a4e5-461e-98a5-c78da1e49b99 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:15.661359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9b2b9-48d4-493d-853a-4e41c865468e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:15.664177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9b2b9-48d4-493d-853a-4e41c865468e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:15.713245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:15.783977 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 338/660] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:37:15.784494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99d4e9-a4e5-461e-98a5-c78da1e49b99 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:15.785703 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea85483-51d9-46d7-92d8-f9a618c18d38 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:15.786172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea85483-51d9-46d7-92d8-f9a618c18d38 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:15.786451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea85483-51d9-46d7-92d8-f9a618c18d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:15.806805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-9ea85483-51d9-46d7-92d8-f9a618c18d38 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:15.808123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9ea85483-51d9-46d7-92d8-f9a618c18d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:15.835228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85209f-106a-4be4-8c4c-0d56c4673dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:15.836030 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85209f-106a-4be4-8c4c-0d56c4673dba None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:15.836440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85209f-106a-4be4-8c4c-0d56c4673dba None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:15.836788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85209f-106a-4be4-8c4c-0d56c4673dba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:15.852278 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85209f-106a-4be4-8c4c-0d56c4673dba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:15.854916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85209f-106a-4be4-8c4c-0d56c4673dba None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:15.869782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-ec85209f-106a-4be4-8c4c-0d56c4673dba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:15.873313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:15.875295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:15.877329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:15.877329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:15.877329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:15.877329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:15.877329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 323/661] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:15.877732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85209f-106a-4be4-8c4c-0d56c4673dba None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:15.897040 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 339/662] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:15.898311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea85483-51d9-46d7-92d8-f9a618c18d38 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:16.013981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea85483-51d9-46d7-92d8-f9a618c18d38 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:16.018240 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:16.029474 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85209f-106a-4be4-8c4c-0d56c4673dba None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:16.056085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af5541-6c41-4784-be81-713fbd522456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:16.056504 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af5541-6c41-4784-be81-713fbd522456 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:16.056803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af5541-6c41-4784-be81-713fbd522456 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:16.057050 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af5541-6c41-4784-be81-713fbd522456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:16.072206 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af5541-6c41-4784-be81-713fbd522456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:16.073777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af5541-6c41-4784-be81-713fbd522456 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:16.080288 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba0972-473e-4572-8758-d194137946a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:16.080923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba0972-473e-4572-8758-d194137946a3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:16.081201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba0972-473e-4572-8758-d194137946a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:16.081476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba0972-473e-4572-8758-d194137946a3 None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:16.082288 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba0972-473e-4572-8758-d194137946a3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:16.082598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba0972-473e-4572-8758-d194137946a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:16.084063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba0972-473e-4572-8758-d194137946a3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-15af5541-6c41-4784-be81-713fbd522456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:16.091356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:16.092390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:16.093378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:16.093378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:16.093378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:16.093378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:16.099508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 340/663] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:16.099508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af5541-6c41-4784-be81-713fbd522456 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:16.106085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 324/664] 10.222.0.86 () {56 vars in 1139 bytes} [Fri Aug 5 19:37:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:16.111048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba0972-473e-4572-8758-d194137946a3 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:16.148819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:16.228008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba0972-473e-4572-8758-d194137946a3 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:16.231154 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af5541-6c41-4784-be81-713fbd522456 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:16.270820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdeb778-7085-4ec3-bcf5-e5de1cd8b6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:16.271558 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb778-7085-4ec3-bcf5-e5de1cd8b6e9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:16.272430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb778-7085-4ec3-bcf5-e5de1cd8b6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:16.272815 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb778-7085-4ec3-bcf5-e5de1cd8b6e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:16.280870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3ab1d-0415-494d-b106-63748eb3ceac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:16.281771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3ab1d-0415-494d-b106-63748eb3ceac None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:16.282125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3ab1d-0415-494d-b106-63748eb3ceac None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:16.282473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3ab1d-0415-494d-b106-63748eb3ceac None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:16.283473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3ab1d-0415-494d-b106-63748eb3ceac None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:16.283867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3ab1d-0415-494d-b106-63748eb3ceac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:16.285315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3ab1d-0415-494d-b106-63748eb3ceac None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:16.289457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdeb778-7085-4ec3-bcf5-e5de1cd8b6e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-ebdeb778-7085-4ec3-bcf5-e5de1cd8b6e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:16.294799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:16.295940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:16.296975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:16.296975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:16.296975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:16.296975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:16.296975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:16.296975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:16.296975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:16.296975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:16.296975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:16.296975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:16.296975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:16.296975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:16.298352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 341/665] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:16.299227 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3ab1d-0415-494d-b106-63748eb3ceac None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:16.299686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 325/666] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:16.300606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdeb778-7085-4ec3-bcf5-e5de1cd8b6e9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:16.314135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:16.422779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdeb778-7085-4ec3-bcf5-e5de1cd8b6e9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:16.423795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:16.434346 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3ab1d-0415-494d-b106-63748eb3ceac None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:16.467583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33ebd6-7518-4e8a-a42d-d04e3486a9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:16.468390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33ebd6-7518-4e8a-a42d-d04e3486a9f2 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:16.468576 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33ebd6-7518-4e8a-a42d-d04e3486a9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:16.468807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33ebd6-7518-4e8a-a42d-d04e3486a9f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:16.473890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cfce7-2b6c-441e-a90a-73ea1f0c7312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:16.475385 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cfce7-2b6c-441e-a90a-73ea1f0c7312 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:16.476454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cfce7-2b6c-441e-a90a-73ea1f0c7312 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:16.476964 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cfce7-2b6c-441e-a90a-73ea1f0c7312 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:16.477888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cfce7-2b6c-441e-a90a-73ea1f0c7312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:16.478403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cfce7-2b6c-441e-a90a-73ea1f0c7312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:16.480010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cfce7-2b6c-441e-a90a-73ea1f0c7312 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:16.497068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 342/667] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:16.498527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cfce7-2b6c-441e-a90a-73ea1f0c7312 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:16.561622 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d33ebd6-7518-4e8a-a42d-d04e3486a9f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:16.563503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d33ebd6-7518-4e8a-a42d-d04e3486a9f2 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:16.575842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:16.628761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cfce7-2b6c-441e-a90a-73ea1f0c7312 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:16.674164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea3920-edab-41d0-bd46-dec52269db66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:16.674943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea3920-edab-41d0-bd46-dec52269db66 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:16.675221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea3920-edab-41d0-bd46-dec52269db66 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:16.675580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea3920-edab-41d0-bd46-dec52269db66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:16.687864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 326/668] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:37:16.687864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33ebd6-7518-4e8a-a42d-d04e3486a9f2 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:16.688376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea3920-edab-41d0-bd46-dec52269db66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:16.688376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea3920-edab-41d0-bd46-dec52269db66 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-71ea3920-edab-41d0-bd46-dec52269db66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:16.703136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:16.708737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:16.710286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:16.710286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:16.710286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:16.710286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:16.710286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:16.710286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:16.710286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:16.710286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:16.710286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 343/669] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:16.710286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea3920-edab-41d0-bd46-dec52269db66 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:16.732848 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:16.843343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33ebd6-7518-4e8a-a42d-d04e3486a9f2 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:16.844463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea3920-edab-41d0-bd46-dec52269db66 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:16.854177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:16.883191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50489715-b468-42b1-83c0-19ca81da199b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:16.883951 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50489715-b468-42b1-83c0-19ca81da199b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:16.884302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50489715-b468-42b1-83c0-19ca81da199b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:16.884699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50489715-b468-42b1-83c0-19ca81da199b None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:16.885144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca059bcd-3770-4b19-b983-7f963f21cd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:16.885618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50489715-b468-42b1-83c0-19ca81da199b None admin] RBAC: Adding query filter params (name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:16.885970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca059bcd-3770-4b19-b983-7f963f21cd24 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:16.886078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50489715-b468-42b1-83c0-19ca81da199b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:16.886368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca059bcd-3770-4b19-b983-7f963f21cd24 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:16.886722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca059bcd-3770-4b19-b983-7f963f21cd24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:16.887813 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50489715-b468-42b1-83c0-19ca81da199b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:16.894113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca059bcd-3770-4b19-b983-7f963f21cd24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:16.895589 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca059bcd-3770-4b19-b983-7f963f21cd24 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:16.898400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 327/670] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:16.899452 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50489715-b468-42b1-83c0-19ca81da199b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-ca059bcd-3770-4b19-b983-7f963f21cd24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:16.918963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:16.920125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:16.921114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:16.921114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:16.921114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:16.921114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:16.921114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:16.921114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:16.921114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:16.921114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:16.921114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 344/671] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:16.921577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca059bcd-3770-4b19-b983-7f963f21cd24 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:17.024512 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:17.030949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50489715-b468-42b1-83c0-19ca81da199b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:17.055798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca059bcd-3770-4b19-b983-7f963f21cd24 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:17.095427 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f41af-6cf1-4d06-8ba3-4e3c8cda7817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:17.096290 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f41af-6cf1-4d06-8ba3-4e3c8cda7817 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:17.096707 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f41af-6cf1-4d06-8ba3-4e3c8cda7817 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:17.097117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f41af-6cf1-4d06-8ba3-4e3c8cda7817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:17.107100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628a657-06f7-4710-9425-5191fcbab564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:17.108183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628a657-06f7-4710-9425-5191fcbab564 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:17.108759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628a657-06f7-4710-9425-5191fcbab564 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:17.109193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628a657-06f7-4710-9425-5191fcbab564 None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:17.109947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628a657-06f7-4710-9425-5191fcbab564 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:17.110248 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628a657-06f7-4710-9425-5191fcbab564 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:17.111849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628a657-06f7-4710-9425-5191fcbab564 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:17.114545 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f41af-6cf1-4d06-8ba3-4e3c8cda7817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:17.117503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f41af-6cf1-4d06-8ba3-4e3c8cda7817 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:17.133423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 345/672] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:17.133423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628a657-06f7-4710-9425-5191fcbab564 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-7d0f41af-6cf1-4d06-8ba3-4e3c8cda7817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:17.143079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:17.144071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:17.144968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:17.144968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:17.144968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:17.144968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:17.144968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 328/673] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:17.145276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f41af-6cf1-4d06-8ba3-4e3c8cda7817 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:17.175616 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:17.269505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f41af-6cf1-4d06-8ba3-4e3c8cda7817 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:17.306744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cccfdd-a3e4-4dc9-864b-84b23624b3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:17.307366 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cccfdd-a3e4-4dc9-864b-84b23624b3dd None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:17.307648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cccfdd-a3e4-4dc9-864b-84b23624b3dd None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:17.307923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cccfdd-a3e4-4dc9-864b-84b23624b3dd None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:17.308675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cccfdd-a3e4-4dc9-864b-84b23624b3dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:17.308980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cccfdd-a3e4-4dc9-864b-84b23624b3dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:17.310429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cccfdd-a3e4-4dc9-864b-84b23624b3dd None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:17.323596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 329/674] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:17.324446 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cccfdd-a3e4-4dc9-864b-84b23624b3dd None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:17.365601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628a657-06f7-4710-9425-5191fcbab564 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:17.400866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f80d3-bad9-40dc-90dd-c41d370c559e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:17.401502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f80d3-bad9-40dc-90dd-c41d370c559e None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:17.401784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f80d3-bad9-40dc-90dd-c41d370c559e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:17.402055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f80d3-bad9-40dc-90dd-c41d370c559e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:17.404788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:17.409125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:17.415029 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f80d3-bad9-40dc-90dd-c41d370c559e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:17.417270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f80d3-bad9-40dc-90dd-c41d370c559e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:17.430806 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cccfdd-a3e4-4dc9-864b-84b23624b3dd None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-7f0f80d3-bad9-40dc-90dd-c41d370c559e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:17.441015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:17.441969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:17.442888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:17.442888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:17.442888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:17.442888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:17.442888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 346/675] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:17.443925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f80d3-bad9-40dc-90dd-c41d370c559e None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:17.473104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7cc43-6198-4418-ba8b-5870c766a27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:17.473718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7cc43-6198-4418-ba8b-5870c766a27b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:17.473987 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7cc43-6198-4418-ba8b-5870c766a27b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:17.474266 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7cc43-6198-4418-ba8b-5870c766a27b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:17.475756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7cc43-6198-4418-ba8b-5870c766a27b None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=afea7d279292424490beff0481ff1512) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:17.476197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7cc43-6198-4418-ba8b-5870c766a27b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:17.477517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7cc43-6198-4418-ba8b-5870c766a27b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:17.483953 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 330/676] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:17 2022] GET /identity/v3/role_assignments?user.id=afea7d279292424490beff0481ff1512&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:17.485207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ac273-d136-4044-9905-6d848c1b4d0f None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:17.485491 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ac273-d136-4044-9905-6d848c1b4d0f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:17.485762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ac273-d136-4044-9905-6d848c1b4d0f None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:17.487012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 331/677] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:17.487722 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ac273-d136-4044-9905-6d848c1b4d0f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:17.595164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f80d3-bad9-40dc-90dd-c41d370c559e None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:17.628265 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ac273-d136-4044-9905-6d848c1b4d0f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:17.640820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7667f-881d-4307-b501-51cdfde91e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:17.641428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7667f-881d-4307-b501-51cdfde91e99 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:17.642224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7667f-881d-4307-b501-51cdfde91e99 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:17.642501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7667f-881d-4307-b501-51cdfde91e99 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:17.643233 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7667f-881d-4307-b501-51cdfde91e99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:17.643534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7667f-881d-4307-b501-51cdfde91e99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:17.644894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7667f-881d-4307-b501-51cdfde91e99 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:17.658129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 347/678] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:17.659662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7667f-881d-4307-b501-51cdfde91e99 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:17.675721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce17bc-f84f-40d2-8d5c-d98125b1c963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:17.676339 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce17bc-f84f-40d2-8d5c-d98125b1c963 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:17.677432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce17bc-f84f-40d2-8d5c-d98125b1c963 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:17.677711 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce17bc-f84f-40d2-8d5c-d98125b1c963 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:17.679091 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce17bc-f84f-40d2-8d5c-d98125b1c963 None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=72d4459164844e718544d7fde333db76) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:17.679455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce17bc-f84f-40d2-8d5c-d98125b1c963 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:17.680931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce17bc-f84f-40d2-8d5c-d98125b1c963 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:17.687633 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 332/679] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:17 2022] GET /identity/v3/role_assignments?user.id=72d4459164844e718544d7fde333db76&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:17.692206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85fbf6-190c-4e63-b3fc-6f59d38ebbf6 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:17.692681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85fbf6-190c-4e63-b3fc-6f59d38ebbf6 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:17.693001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85fbf6-190c-4e63-b3fc-6f59d38ebbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:17.718697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-5f85fbf6-190c-4e63-b3fc-6f59d38ebbf6 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:17.720011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5f85fbf6-190c-4e63-b3fc-6f59d38ebbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:17.725609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:17.739029 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:17.802567 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7667f-881d-4307-b501-51cdfde91e99 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:17.828380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 333/680] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:17.829916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85fbf6-190c-4e63-b3fc-6f59d38ebbf6 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:17.863125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc8ad3-c20d-4cd4-b230-165a2aff5fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:17.863756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc8ad3-c20d-4cd4-b230-165a2aff5fec None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:17.864038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc8ad3-c20d-4cd4-b230-165a2aff5fec None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:17.864312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc8ad3-c20d-4cd4-b230-165a2aff5fec None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:17.865017 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc8ad3-c20d-4cd4-b230-165a2aff5fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:17.865322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc8ad3-c20d-4cd4-b230-165a2aff5fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:17.868548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc8ad3-c20d-4cd4-b230-165a2aff5fec None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:17.883338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 348/681] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:17.884149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc8ad3-c20d-4cd4-b230-165a2aff5fec None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:17.985583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85fbf6-190c-4e63-b3fc-6f59d38ebbf6 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:18.023981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc8ad3-c20d-4cd4-b230-165a2aff5fec None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:18.035849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:18.047273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b7025-ed35-40d0-8503-e0d62c95cefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:18.047928 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b7025-ed35-40d0-8503-e0d62c95cefa None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:18.048202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b7025-ed35-40d0-8503-e0d62c95cefa None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:18.048469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b7025-ed35-40d0-8503-e0d62c95cefa None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:18.058794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b7025-ed35-40d0-8503-e0d62c95cefa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:18.060508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b7025-ed35-40d0-8503-e0d62c95cefa None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:18.064657 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:18.068068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b1575-1023-47cf-aea9-b74525f6961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:18.069927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b1575-1023-47cf-aea9-b74525f6961d None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:18.070394 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b1575-1023-47cf-aea9-b74525f6961d None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:18.070875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90b1575-1023-47cf-aea9-b74525f6961d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-507b7025-ed35-40d0-8503-e0d62c95cefa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:18.079070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:18.080051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:18.081016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:18.081016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:18.081016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:18.081016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:18.081016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:18.081016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:18.081016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:18.081016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:18.081016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 334/682] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:18.081516 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b7025-ed35-40d0-8503-e0d62c95cefa None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:18.095048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90b1575-1023-47cf-aea9-b74525f6961d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-f90b1575-1023-47cf-aea9-b74525f6961d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:18.101484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:18.102573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:18.103529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:18.103529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:18.103529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:18.103529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:18.103529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:18.103529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:18.103529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:18.103529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:18.103529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:18.103529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:18.103529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:37:18.103529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:18.109866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 349/683] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:37:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:18.109866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b1575-1023-47cf-aea9-b74525f6961d None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:18.235265 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b7025-ed35-40d0-8503-e0d62c95cefa None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:18.249206 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90b1575-1023-47cf-aea9-b74525f6961d None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:18.297354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ea7b0-bf76-46c3-8eb9-5c4dbb0601ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:18.297985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ea7b0-bf76-46c3-8eb9-5c4dbb0601ba None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:18.298261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ea7b0-bf76-46c3-8eb9-5c4dbb0601ba None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:18.298531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ea7b0-bf76-46c3-8eb9-5c4dbb0601ba None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:18.299427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ea7b0-bf76-46c3-8eb9-5c4dbb0601ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:18.299734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ea7b0-bf76-46c3-8eb9-5c4dbb0601ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:18.301299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ea7b0-bf76-46c3-8eb9-5c4dbb0601ba None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:18.322845 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 350/684] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:18.322845 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ea7b0-bf76-46c3-8eb9-5c4dbb0601ba None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:18.324217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8979bbe-232d-44ae-bad2-7c91131389cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:18.324803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8979bbe-232d-44ae-bad2-7c91131389cc None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:18.325084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8979bbe-232d-44ae-bad2-7c91131389cc None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:18.325352 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8979bbe-232d-44ae-bad2-7c91131389cc None admin] PATH_INFO: `/v3/users/swift` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:18.335111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8979bbe-232d-44ae-bad2-7c91131389cc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:18.337175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8979bbe-232d-44ae-bad2-7c91131389cc None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-b8979bbe-232d-44ae-bad2-7c91131389cc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:18.351560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:18.353329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:18.355554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:18.355554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:18.355554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:18.355554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:18.355554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:18.355554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:18.355554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:18.355554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:18.355554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:37:18.355554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:18.355554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 335/685] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:37:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:18.355554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8979bbe-232d-44ae-bad2-7c91131389cc None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:18.391654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:18.437618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:18.495301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8979bbe-232d-44ae-bad2-7c91131389cc None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:18.514923 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ea7b0-bf76-46c3-8eb9-5c4dbb0601ba None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:18.537318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bc9cd-54ba-4506-aff0-46d01588c229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:18.537948 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bc9cd-54ba-4506-aff0-46d01588c229 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:18.538225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bc9cd-54ba-4506-aff0-46d01588c229 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:18.538500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0bc9cd-54ba-4506-aff0-46d01588c229 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:18.539428 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bc9cd-54ba-4506-aff0-46d01588c229 None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=80097afc3ce540e6a8060f696535e62c) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:18.539797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bc9cd-54ba-4506-aff0-46d01588c229 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:18.541237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0bc9cd-54ba-4506-aff0-46d01588c229 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:18.555008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 336/686] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:18 2022] GET /identity/v3/role_assignments?user.id=80097afc3ce540e6a8060f696535e62c&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:18.555008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bc9cd-54ba-4506-aff0-46d01588c229 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:18.575434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d475ca1-a65a-4502-80d8-9f844919563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:18.576472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d475ca1-a65a-4502-80d8-9f844919563a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:18.576967 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d475ca1-a65a-4502-80d8-9f844919563a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:18.577477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d475ca1-a65a-4502-80d8-9f844919563a None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:18.578494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d475ca1-a65a-4502-80d8-9f844919563a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:18.579103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d475ca1-a65a-4502-80d8-9f844919563a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:18.580746 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d475ca1-a65a-4502-80d8-9f844919563a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:18.613071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 351/687] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:37:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:18.623745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d475ca1-a65a-4502-80d8-9f844919563a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:18.729200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0bc9cd-54ba-4506-aff0-46d01588c229 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:18.760000 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:18.779434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654732b-8fa5-4d74-9614-2aaab3d82e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:18.780342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654732b-8fa5-4d74-9614-2aaab3d82e3d None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:18.780776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654732b-8fa5-4d74-9614-2aaab3d82e3d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:18.781174 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654732b-8fa5-4d74-9614-2aaab3d82e3d None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:18.782223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654732b-8fa5-4d74-9614-2aaab3d82e3d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:18.783178 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654732b-8fa5-4d74-9614-2aaab3d82e3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:18.784798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654732b-8fa5-4d74-9614-2aaab3d82e3d None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:18.806976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 337/688] 10.222.0.86 () {56 vars in 1135 bytes} [Fri Aug 5 19:37:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:18.813282 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654732b-8fa5-4d74-9614-2aaab3d82e3d None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:18.821074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:18.839519 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d475ca1-a65a-4502-80d8-9f844919563a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:18.891106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39fb50-f7a9-4090-b759-685703a6968f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:18.892218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39fb50-f7a9-4090-b759-685703a6968f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:18.892707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39fb50-f7a9-4090-b759-685703a6968f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:18.893147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39fb50-f7a9-4090-b759-685703a6968f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:18.906951 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39fb50-f7a9-4090-b759-685703a6968f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:18.909125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39fb50-f7a9-4090-b759-685703a6968f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-8e39fb50-f7a9-4090-b759-685703a6968f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:18.927687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:18.929176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:18.930090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:18.930090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:18.930090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:18.930090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:18.932932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 352/689] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:18.940768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39fb50-f7a9-4090-b759-685703a6968f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:18.952041 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654732b-8fa5-4d74-9614-2aaab3d82e3d None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:18.995075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358f57b-d84e-43f4-9572-1df1926b33fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:18.996127 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358f57b-d84e-43f4-9572-1df1926b33fc None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:18.996649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358f57b-d84e-43f4-9572-1df1926b33fc None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:18.997142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358f57b-d84e-43f4-9572-1df1926b33fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:19.079946 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39fb50-f7a9-4090-b759-685703a6968f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:19.096695 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:19.141214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358f57b-d84e-43f4-9572-1df1926b33fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:19.146667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0c674-d35d-48bf-937d-48d56970b28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:19.148761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358f57b-d84e-43f4-9572-1df1926b33fc None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:19.149695 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0c674-d35d-48bf-937d-48d56970b28f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:19.152593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0c674-d35d-48bf-937d-48d56970b28f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:19.153219 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0c674-d35d-48bf-937d-48d56970b28f None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:19.154170 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0c674-d35d-48bf-937d-48d56970b28f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:19.154700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0c674-d35d-48bf-937d-48d56970b28f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:19.156337 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0c674-d35d-48bf-937d-48d56970b28f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:19.186339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 353/690] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:19.192348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0c674-d35d-48bf-937d-48d56970b28f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:19.195642 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:19.306491 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 338/691] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:37:19.310731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0c674-d35d-48bf-937d-48d56970b28f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:19.312392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358f57b-d84e-43f4-9572-1df1926b33fc None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:19.349434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e91a0f-446c-4942-a9e2-a52c113dd2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:19.350592 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e91a0f-446c-4942-a9e2-a52c113dd2d5 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:19.350899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e91a0f-446c-4942-a9e2-a52c113dd2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:19.351281 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e91a0f-446c-4942-a9e2-a52c113dd2d5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:19.359090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e91a0f-446c-4942-a9e2-a52c113dd2d5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:19.360545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e91a0f-446c-4942-a9e2-a52c113dd2d5 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-c6e91a0f-446c-4942-a9e2-a52c113dd2d5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:19.375771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:19.376810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:19.378602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:19.378602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:19.378602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:19.378602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:19.378602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:19.378602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:19.378602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:19.378602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:19.378602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:37:19.378602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:19.378602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 354/692] 10.222.0.86 () {56 vars in 1093 bytes} [Fri Aug 5 19:37:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:19.384319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e91a0f-446c-4942-a9e2-a52c113dd2d5 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:19.426948 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:19.458885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358f57b-d84e-43f4-9572-1df1926b33fc None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:19.482245 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:19.515643 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f05d3-1dbb-44e1-9706-4bb3978008b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:19.516383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f05d3-1dbb-44e1-9706-4bb3978008b0 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:19.516658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f05d3-1dbb-44e1-9706-4bb3978008b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:19.516923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f05d3-1dbb-44e1-9706-4bb3978008b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:19.528985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f05d3-1dbb-44e1-9706-4bb3978008b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:19.530045 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e91a0f-446c-4942-a9e2-a52c113dd2d5 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:19.530452 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f05d3-1dbb-44e1-9706-4bb3978008b0 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-ef9f05d3-1dbb-44e1-9706-4bb3978008b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:19.549656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:19.550807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:19.551906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:19.551906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:19.551906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:19.551906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:19.551906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 339/693] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:19.559740 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f05d3-1dbb-44e1-9706-4bb3978008b0 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:19.575777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa50639-5bcd-4595-a2c8-c0ef0793a24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:19.576413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa50639-5bcd-4595-a2c8-c0ef0793a24a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:19.576691 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa50639-5bcd-4595-a2c8-c0ef0793a24a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:19.576960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa50639-5bcd-4595-a2c8-c0ef0793a24a None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:19.577769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa50639-5bcd-4595-a2c8-c0ef0793a24a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:19.578078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa50639-5bcd-4595-a2c8-c0ef0793a24a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:19.579692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa50639-5bcd-4595-a2c8-c0ef0793a24a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:19.617916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 355/694] 10.222.0.86 () {56 vars in 1133 bytes} [Fri Aug 5 19:37:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:19.628819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa50639-5bcd-4595-a2c8-c0ef0793a24a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:19.769319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa50639-5bcd-4595-a2c8-c0ef0793a24a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:19.790639 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f05d3-1dbb-44e1-9706-4bb3978008b0 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:19.803077 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:19.813497 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc8c29-d2ab-4da4-94e0-3edbd21ced8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:19.814367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc8c29-d2ab-4da4-94e0-3edbd21ced8c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:19.814871 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc8c29-d2ab-4da4-94e0-3edbd21ced8c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:19.815213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc8c29-d2ab-4da4-94e0-3edbd21ced8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:19.872155 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db7f2f-ffc9-43a2-8dec-f582aa329f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:19.872710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db7f2f-ffc9-43a2-8dec-f582aa329f6a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:19.872989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db7f2f-ffc9-43a2-8dec-f582aa329f6a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:19.873265 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db7f2f-ffc9-43a2-8dec-f582aa329f6a None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:19.874033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db7f2f-ffc9-43a2-8dec-f582aa329f6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:19.874341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db7f2f-ffc9-43a2-8dec-f582aa329f6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:19.875768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db7f2f-ffc9-43a2-8dec-f582aa329f6a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:19.877481 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:19.901534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 340/695] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:19.901534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1405f94-7994-46b0-930f-f59d429e555e None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:19.901534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1405f94-7994-46b0-930f-f59d429e555e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:19.901534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1405f94-7994-46b0-930f-f59d429e555e None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:19.904205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 341/696] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:19.904740 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1405f94-7994-46b0-930f-f59d429e555e None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:19.907433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc8c29-d2ab-4da4-94e0-3edbd21ced8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:19.908908 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc8c29-d2ab-4da4-94e0-3edbd21ced8c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:20.045586 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 356/697] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:37:20.049027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1405f94-7994-46b0-930f-f59d429e555e None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:20.051178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943622f-e8e4-4203-adfb-9f84d8f00819 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:20.051521 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943622f-e8e4-4203-adfb-9f84d8f00819 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:20.051807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943622f-e8e4-4203-adfb-9f84d8f00819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:20.074724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-1943622f-e8e4-4203-adfb-9f84d8f00819 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:20.074724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1943622f-e8e4-4203-adfb-9f84d8f00819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:20.097030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:20.105025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096e87b-d919-48cf-be67-93830c06608b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:20.106068 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096e87b-d919-48cf-be67-93830c06608b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:20.106493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096e87b-d919-48cf-be67-93830c06608b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:20.107074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096e87b-d919-48cf-be67-93830c06608b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:20.116810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096e87b-d919-48cf-be67-93830c06608b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:20.118795 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096e87b-d919-48cf-be67-93830c06608b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-a096e87b-d919-48cf-be67-93830c06608b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:20.136253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:20.137381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:20.138537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:20.138537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:20.138537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:20.138537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:20.138537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:20.138537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:20.138537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:20.138537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:20.140830 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 342/698] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:20.141897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096e87b-d919-48cf-be67-93830c06608b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:20.160008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 357/699] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:20.160886 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943622f-e8e4-4203-adfb-9f84d8f00819 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:20.216568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:20.269120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096e87b-d919-48cf-be67-93830c06608b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:20.288128 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943622f-e8e4-4203-adfb-9f84d8f00819 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:20.317234 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017bc84-48f3-4e8b-9d5b-f8cf380a929e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:20.318095 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017bc84-48f3-4e8b-9d5b-f8cf380a929e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:20.319659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017bc84-48f3-4e8b-9d5b-f8cf380a929e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:20.320083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017bc84-48f3-4e8b-9d5b-f8cf380a929e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:20.329375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f16f0-f7c1-42f3-a8f6-1efdb378c8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:20.330002 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f16f0-f7c1-42f3-a8f6-1efdb378c8a8 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:20.330281 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f16f0-f7c1-42f3-a8f6-1efdb378c8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:20.330549 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f16f0-f7c1-42f3-a8f6-1efdb378c8a8 None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:20.331348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f16f0-f7c1-42f3-a8f6-1efdb378c8a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:20.331648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f16f0-f7c1-42f3-a8f6-1efdb378c8a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:20.333117 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f16f0-f7c1-42f3-a8f6-1efdb378c8a8 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:20.335625 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017bc84-48f3-4e8b-9d5b-f8cf380a929e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:20.337836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017bc84-48f3-4e8b-9d5b-f8cf380a929e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:20.348552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 358/700] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:20.349397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f16f0-f7c1-42f3-a8f6-1efdb378c8a8 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-a017bc84-48f3-4e8b-9d5b-f8cf380a929e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:20.361838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:20.362819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:20.363751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:20.363751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:20.363751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:20.363751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:20.363751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 343/701] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:20.366811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400dbd-6620-47c5-963f-79d87349bce3 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:20.367416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400dbd-6620-47c5-963f-79d87349bce3 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:20.368189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:20.369352 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400dbd-6620-47c5-963f-79d87349bce3 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:20.371386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 344/702] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:20.371386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78400dbd-6620-47c5-963f-79d87349bce3 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:20.491162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f16f0-f7c1-42f3-a8f6-1efdb378c8a8 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:20.498333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78400dbd-6620-47c5-963f-79d87349bce3 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:20.507279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:20.544738 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b2612-c6ab-48b1-9c59-07aded8a05cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:20.545072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6ffab-eb2d-4f15-8ebe-1fed0e917abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:20.545504 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b2612-c6ab-48b1-9c59-07aded8a05cd None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:20.545750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b2612-c6ab-48b1-9c59-07aded8a05cd None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:20.545851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6ffab-eb2d-4f15-8ebe-1fed0e917abd None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:20.546000 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6ffab-eb2d-4f15-8ebe-1fed0e917abd None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:20.546104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b2612-c6ab-48b1-9c59-07aded8a05cd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:20.546249 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6ffab-eb2d-4f15-8ebe-1fed0e917abd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:20.546590 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b2612-c6ab-48b1-9c59-07aded8a05cd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:20.547030 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6ffab-eb2d-4f15-8ebe-1fed0e917abd None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=e8adb0fd1fbc4268b70f3936036e550b) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:20.547333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6ffab-eb2d-4f15-8ebe-1fed0e917abd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:20.547987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b2612-c6ab-48b1-9c59-07aded8a05cd None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:20.548761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6ffab-eb2d-4f15-8ebe-1fed0e917abd None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:20.566952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 345/703] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:20 2022] GET /identity/v3/role_assignments?user.id=e8adb0fd1fbc4268b70f3936036e550b&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:20.566952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6ffab-eb2d-4f15-8ebe-1fed0e917abd None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-339b2612-c6ab-48b1-9c59-07aded8a05cd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:20.567209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:20.568123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:20.569083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 359/704] 10.222.0.86 () {56 vars in 1129 bytes} [Fri Aug 5 19:37:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:20.569931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e584b-b88d-4293-a91b-eb16409bc16c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:20.569931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e584b-b88d-4293-a91b-eb16409bc16c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:20.569931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e584b-b88d-4293-a91b-eb16409bc16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:20.588847 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-908e584b-b88d-4293-a91b-eb16409bc16c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:20.590254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-908e584b-b88d-4293-a91b-eb16409bc16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:20.690846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 360/705] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:20.691806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e584b-b88d-4293-a91b-eb16409bc16c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:20.704903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:20.734899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6ffab-eb2d-4f15-8ebe-1fed0e917abd None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:20.798954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8372c-7b32-46c5-b1f8-d37ae08b57e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:20.799676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8372c-7b32-46c5-b1f8-d37ae08b57e7 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:20.799951 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8372c-7b32-46c5-b1f8-d37ae08b57e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:20.800218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8372c-7b32-46c5-b1f8-d37ae08b57e7 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:20.800903 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8372c-7b32-46c5-b1f8-d37ae08b57e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:20.801876 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8372c-7b32-46c5-b1f8-d37ae08b57e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:20.802553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8372c-7b32-46c5-b1f8-d37ae08b57e7 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:20.816674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 346/706] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:20.817525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8372c-7b32-46c5-b1f8-d37ae08b57e7 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:20.852894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e584b-b88d-4293-a91b-eb16409bc16c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:20.864979 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:20.904913 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a98e35-4e2f-4de8-8d41-5af3acdb4b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:20.905549 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a98e35-4e2f-4de8-8d41-5af3acdb4b4e None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:20.905829 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a98e35-4e2f-4de8-8d41-5af3acdb4b4e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:20.906095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a98e35-4e2f-4de8-8d41-5af3acdb4b4e None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:20.906808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a98e35-4e2f-4de8-8d41-5af3acdb4b4e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:20.907112 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a98e35-4e2f-4de8-8d41-5af3acdb4b4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:20.911659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a98e35-4e2f-4de8-8d41-5af3acdb4b4e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:20.919023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 361/707] 10.222.0.86 () {56 vars in 1124 bytes} [Fri Aug 5 19:37:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:20.920457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a98e35-4e2f-4de8-8d41-5af3acdb4b4e None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:20.964237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8372c-7b32-46c5-b1f8-d37ae08b57e7 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:21.008889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1ade7-a197-4881-be25-c6097c47f5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:21.009760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1ade7-a197-4881-be25-c6097c47f5fc None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:21.010193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1ade7-a197-4881-be25-c6097c47f5fc None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:21.010519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1ade7-a197-4881-be25-c6097c47f5fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:21.012299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:21.024150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1ade7-a197-4881-be25-c6097c47f5fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:21.026641 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a1ade7-a197-4881-be25-c6097c47f5fc None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-53a1ade7-a197-4881-be25-c6097c47f5fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:21.043789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:21.044854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:21.045782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:21.045782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:21.045782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:21.045782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:21.045782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 347/708] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:21.047216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1ade7-a197-4881-be25-c6097c47f5fc None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:21.062886 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a98e35-4e2f-4de8-8d41-5af3acdb4b4e None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:21.120136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee62b9-1592-4317-a603-1540cf920a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:21.121796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee62b9-1592-4317-a603-1540cf920a19 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:21.122392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee62b9-1592-4317-a603-1540cf920a19 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:21.122870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee62b9-1592-4317-a603-1540cf920a19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:21.134200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee62b9-1592-4317-a603-1540cf920a19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:21.135997 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee62b9-1592-4317-a603-1540cf920a19 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:21.148149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-34ee62b9-1592-4317-a603-1540cf920a19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:21.155190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:21.156158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:21.157084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:21.157084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:21.157084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:21.157084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:21.157084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:21.157084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:21.157084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:21.157084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:21.164572 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 362/709] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:21.164572 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee62b9-1592-4317-a603-1540cf920a19 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:21.179192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1ade7-a197-4881-be25-c6097c47f5fc None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:21.228474 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674b769-5191-4ec9-911d-3d90ab43f3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:21.229682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674b769-5191-4ec9-911d-3d90ab43f3de None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:21.230510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674b769-5191-4ec9-911d-3d90ab43f3de None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:21.231151 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674b769-5191-4ec9-911d-3d90ab43f3de None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:21.232139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674b769-5191-4ec9-911d-3d90ab43f3de None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:21.232817 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674b769-5191-4ec9-911d-3d90ab43f3de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:21.235070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674b769-5191-4ec9-911d-3d90ab43f3de None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:21.243901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 348/710] 10.222.0.86 () {56 vars in 1124 bytes} [Fri Aug 5 19:37:21 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:21.245039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674b769-5191-4ec9-911d-3d90ab43f3de None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:21.292611 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee62b9-1592-4317-a603-1540cf920a19 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:21.332816 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:21.346456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07635f5c-41d9-4da0-b1c8-df2d8ac7a780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:21.347362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07635f5c-41d9-4da0-b1c8-df2d8ac7a780 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:21.347638 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07635f5c-41d9-4da0-b1c8-df2d8ac7a780 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:21.347902 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07635f5c-41d9-4da0-b1c8-df2d8ac7a780 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:21.348611 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07635f5c-41d9-4da0-b1c8-df2d8ac7a780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:21.348911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07635f5c-41d9-4da0-b1c8-df2d8ac7a780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:21.351095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07635f5c-41d9-4da0-b1c8-df2d8ac7a780 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:21.368915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 363/711] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:21.370497 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ced36b-c725-4ac1-bb59-3fd07657ce6a None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:21.370830 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ced36b-c725-4ac1-bb59-3fd07657ce6a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:21.371461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ced36b-c725-4ac1-bb59-3fd07657ce6a None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:21.377367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 364/712] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:21.377367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ced36b-c725-4ac1-bb59-3fd07657ce6a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:21.397249 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674b769-5191-4ec9-911d-3d90ab43f3de None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:21.454682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e993901-093f-41c7-b2d3-4825b3cff487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:21.455843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e993901-093f-41c7-b2d3-4825b3cff487 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:21.456928 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e993901-093f-41c7-b2d3-4825b3cff487 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:21.457394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e993901-093f-41c7-b2d3-4825b3cff487 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:21.458611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e993901-093f-41c7-b2d3-4825b3cff487 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:21.459695 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e993901-093f-41c7-b2d3-4825b3cff487 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:21.462345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e993901-093f-41c7-b2d3-4825b3cff487 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:21.481898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 349/713] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:21.486898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4814570-44fd-458a-817e-739b96430f25 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:21.486898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4814570-44fd-458a-817e-739b96430f25 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:21.486898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4814570-44fd-458a-817e-739b96430f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:21.517577 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-a4814570-44fd-458a-817e-739b96430f25 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:21.518928 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a4814570-44fd-458a-817e-739b96430f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:21.522972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:21.549350 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ced36b-c725-4ac1-bb59-3fd07657ce6a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:21.598320 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940ccc1-8ac9-41f5-afe5-5169be687335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:21.598989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940ccc1-8ac9-41f5-afe5-5169be687335 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:21.599261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940ccc1-8ac9-41f5-afe5-5169be687335 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:21.599534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940ccc1-8ac9-41f5-afe5-5169be687335 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:21.601776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940ccc1-8ac9-41f5-afe5-5169be687335 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:21.603766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940ccc1-8ac9-41f5-afe5-5169be687335 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:21.622962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 350/714] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:21.624784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4814570-44fd-458a-817e-739b96430f25 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:21.633486 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 365/715] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:37:21 2022] POST /identity/v3/users => generated 254 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:37:21.657729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940ccc1-8ac9-41f5-afe5-5169be687335 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:21.765805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:21.794148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4814570-44fd-458a-817e-739b96430f25 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:21.834801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3cf6c-c248-4599-a208-cf281cff7dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:21.835885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3cf6c-c248-4599-a208-cf281cff7dba None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:21.836520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3cf6c-c248-4599-a208-cf281cff7dba None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:21.836864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3cf6c-c248-4599-a208-cf281cff7dba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:21.837761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3cf6c-c248-4599-a208-cf281cff7dba None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=ab8f9d9cf4ca4e38ae6eb67e949b40fd) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:21.838140 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3cf6c-c248-4599-a208-cf281cff7dba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:21.839701 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3cf6c-c248-4599-a208-cf281cff7dba None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:21.844431 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:21.852544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 351/716] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:21 2022] GET /identity/v3/role_assignments?user.id=ab8f9d9cf4ca4e38ae6eb67e949b40fd&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=93eb204498fc49d297bcd1841e00818d => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:21.904761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940ccc1-8ac9-41f5-afe5-5169be687335 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:21.961804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573f4be-435a-42fb-bd2e-4da790bdb1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:21.962751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573f4be-435a-42fb-bd2e-4da790bdb1ba None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:21.962909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573f4be-435a-42fb-bd2e-4da790bdb1ba None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:21.963220 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573f4be-435a-42fb-bd2e-4da790bdb1ba None admin] PATH_INFO: `/v3/services/network` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:21.963741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8573f4be-435a-42fb-bd2e-4da790bdb1ba None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:21.965145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8573f4be-435a-42fb-bd2e-4da790bdb1ba None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-8573f4be-435a-42fb-bd2e-4da790bdb1ba None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:21.980748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:21.981835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:21.983142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 366/717] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:21 2022] GET /identity/v3/services/network => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:21.988974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3cf6c-c248-4599-a208-cf281cff7dba None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:22.140068 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3cf6c-c248-4599-a208-cf281cff7dba None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:22.190259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52aa8a8-d357-4026-8f36-50435cc1ec76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:22.190259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aa8a8-d357-4026-8f36-50435cc1ec76 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:22.190259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aa8a8-d357-4026-8f36-50435cc1ec76 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:22.190259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52aa8a8-d357-4026-8f36-50435cc1ec76 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:22.190952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52aa8a8-d357-4026-8f36-50435cc1ec76 None admin] RBAC: Adding query filter params (name=network) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:22.190952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52aa8a8-d357-4026-8f36-50435cc1ec76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:22.190952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52aa8a8-d357-4026-8f36-50435cc1ec76 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:22.199087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 352/718] 10.222.0.86 () {56 vars in 1112 bytes} [Fri Aug 5 19:37:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:22.205064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573f4be-435a-42fb-bd2e-4da790bdb1ba None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:22.369578 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573f4be-435a-42fb-bd2e-4da790bdb1ba None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:22.387055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:22.417856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a985d7f-ed22-4943-8c11-47c253317364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:22.418397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a985d7f-ed22-4943-8c11-47c253317364 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:22.418698 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a985d7f-ed22-4943-8c11-47c253317364 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:22.418974 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a985d7f-ed22-4943-8c11-47c253317364 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:22.419664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a985d7f-ed22-4943-8c11-47c253317364 None admin] RBAC: Adding query filter params (type=network) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:22.419968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a985d7f-ed22-4943-8c11-47c253317364 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:22.421464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a985d7f-ed22-4943-8c11-47c253317364 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:22.432590 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 367/719] 10.222.0.86 () {56 vars in 1112 bytes} [Fri Aug 5 19:37:22 2022] GET /identity/v3/services?type=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:23.141762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae9d0f-6a46-45ed-aba1-fa843e05e292 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:23.142293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae9d0f-6a46-45ed-aba1-fa843e05e292 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:23.142293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae9d0f-6a46-45ed-aba1-fa843e05e292 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:23.143817 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 353/720] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:23.150605 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ed4e-1dd2-4e58-a764-c936831f58c9 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:23.151128 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ed4e-1dd2-4e58-a764-c936831f58c9 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:23.151256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ed4e-1dd2-4e58-a764-c936831f58c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:23.176886 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-2578ed4e-1dd2-4e58-a764-c936831f58c9 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:23.178721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2578ed4e-1dd2-4e58-a764-c936831f58c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:23.267298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 368/721] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:23.293524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae9d0f-6a46-45ed-aba1-fa843e05e292 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:23.418764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:23.492351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae9d0f-6a46-45ed-aba1-fa843e05e292 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:23.577614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50f907-5328-44ec-8335-d9b3ac4a9855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:23.578192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50f907-5328-44ec-8335-d9b3ac4a9855 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:23.578468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50f907-5328-44ec-8335-d9b3ac4a9855 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:23.578759 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50f907-5328-44ec-8335-d9b3ac4a9855 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:23.601320 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50f907-5328-44ec-8335-d9b3ac4a9855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-ce50f907-5328-44ec-8335-d9b3ac4a9855 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:23.615983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:23.617099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:23.617994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:23.617994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:23.617994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:23.617994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:23.617994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:23.617994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:23.617994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:23.617994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:23.617994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:23.617994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:23.617994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:37:23.617994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:23.620860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 354/722] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:37:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:23.627750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578ed4e-1dd2-4e58-a764-c936831f58c9 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:23.783710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578ed4e-1dd2-4e58-a764-c936831f58c9 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:23.786314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:23.835890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b717965-f283-46e6-bd5d-88db3d3e0934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:23.836956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b717965-f283-46e6-bd5d-88db3d3e0934 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:23.837443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b717965-f283-46e6-bd5d-88db3d3e0934 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:23.837947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b717965-f283-46e6-bd5d-88db3d3e0934 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:23.838943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b717965-f283-46e6-bd5d-88db3d3e0934 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:23.839472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b717965-f283-46e6-bd5d-88db3d3e0934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:23.841076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b717965-f283-46e6-bd5d-88db3d3e0934 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:23.851859 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 369/723] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:37:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:23.859643 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50f907-5328-44ec-8335-d9b3ac4a9855 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:23.928229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492672e-be73-463d-bb52-b0ab03d0f9ec None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:23.928873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492672e-be73-463d-bb52-b0ab03d0f9ec None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:23.929409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492672e-be73-463d-bb52-b0ab03d0f9ec None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:23.931657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 370/724] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:23.936988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c52a1-21a7-4dfb-85a1-2fff39fa4911 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:23.937543 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c52a1-21a7-4dfb-85a1-2fff39fa4911 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:23.938387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c52a1-21a7-4dfb-85a1-2fff39fa4911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:23.974274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-d83c52a1-21a7-4dfb-85a1-2fff39fa4911 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:23.976059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d83c52a1-21a7-4dfb-85a1-2fff39fa4911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:23.981507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50f907-5328-44ec-8335-d9b3ac4a9855 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:24.007371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:24.029588 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fb3fa-92c5-49ac-884d-327e7b0ec48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:24.030240 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fb3fa-92c5-49ac-884d-327e7b0ec48d None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:24.030512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fb3fa-92c5-49ac-884d-327e7b0ec48d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:24.030805 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fb3fa-92c5-49ac-884d-327e7b0ec48d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:24.051004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587fb3fa-92c5-49ac-884d-327e7b0ec48d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:24.052773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587fb3fa-92c5-49ac-884d-327e7b0ec48d None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-587fb3fa-92c5-49ac-884d-327e7b0ec48d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:24.083012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:24.084635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:24.085685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:24.085685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:24.085685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:24.085685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:24.089441 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 355/725] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:24.095760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fb3fa-92c5-49ac-884d-327e7b0ec48d None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:24.112749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 371/726] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:24.152620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c52a1-21a7-4dfb-85a1-2fff39fa4911 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:24.222475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fb3fa-92c5-49ac-884d-327e7b0ec48d None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:24.268196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13933747-18d2-4f4d-84c3-738130a435d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:24.269243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13933747-18d2-4f4d-84c3-738130a435d4 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:24.270393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13933747-18d2-4f4d-84c3-738130a435d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:24.271029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13933747-18d2-4f4d-84c3-738130a435d4 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:24.275839 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13933747-18d2-4f4d-84c3-738130a435d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:24.276384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13933747-18d2-4f4d-84c3-738130a435d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:24.277939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13933747-18d2-4f4d-84c3-738130a435d4 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:24.304129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 356/727] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:24.311731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13933747-18d2-4f4d-84c3-738130a435d4 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:24.353746 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:24.371318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:24.378872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c52a1-21a7-4dfb-85a1-2fff39fa4911 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:24.448713 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13933747-18d2-4f4d-84c3-738130a435d4 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:24.466836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13114171-c115-4241-917f-3cd1750107bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:24.467462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13114171-c115-4241-917f-3cd1750107bd None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:24.467742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13114171-c115-4241-917f-3cd1750107bd None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:24.468015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13114171-c115-4241-917f-3cd1750107bd None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:24.468833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13114171-c115-4241-917f-3cd1750107bd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:24.470330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13114171-c115-4241-917f-3cd1750107bd None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:24.489610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c801c3-c91f-4079-b64e-b86ca814bf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:24.490773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c801c3-c91f-4079-b64e-b86ca814bf05 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:24.491136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c801c3-c91f-4079-b64e-b86ca814bf05 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:24.491415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c801c3-c91f-4079-b64e-b86ca814bf05 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:24.496536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 372/728] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:37:24 2022] POST /identity/v3/services => generated 251 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:37:24.498339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3790a6e-fb1d-47ed-80cb-6c8ccb606320 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:24.499041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3790a6e-fb1d-47ed-80cb-6c8ccb606320 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:24.499591 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3790a6e-fb1d-47ed-80cb-6c8ccb606320 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:24.504703 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 373/729] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:24.508024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c801c3-c91f-4079-b64e-b86ca814bf05 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:24.509604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c801c3-c91f-4079-b64e-b86ca814bf05 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:24.515491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683eb78c-33a5-40b9-9ff0-7ffb543b1b16 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:24.516102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683eb78c-33a5-40b9-9ff0-7ffb543b1b16 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:24.516552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683eb78c-33a5-40b9-9ff0-7ffb543b1b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-06c801c3-c91f-4079-b64e-b86ca814bf05 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:24.539674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:24.540733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:24.541673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:24.541673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:24.541673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:24.541673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:24.541673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:24.541673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:24.541673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:24.541673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:24.541673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:37:24.541673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:24.545127 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 357/730] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:37:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:24.551952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c801c3-c91f-4079-b64e-b86ca814bf05 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:24.562094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-683eb78c-33a5-40b9-9ff0-7ffb543b1b16 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:24.563839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-683eb78c-33a5-40b9-9ff0-7ffb543b1b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:24.680764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:24.689319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c801c3-c91f-4079-b64e-b86ca814bf05 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:24.760954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 374/731] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:24.760954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336be05-ac44-45e4-b949-6ed038286dda None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:24.760954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336be05-ac44-45e4-b949-6ed038286dda None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:24.760954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336be05-ac44-45e4-b949-6ed038286dda None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:24.762881 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 375/732] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:24.767419 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46edf23-e7fb-4c2e-8e1a-c7c368c5969b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:24.767629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808677e3-b5ca-4717-b902-0289b9ab02ff None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:24.767713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808677e3-b5ca-4717-b902-0289b9ab02ff None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:24.767982 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808677e3-b5ca-4717-b902-0289b9ab02ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:24.768082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46edf23-e7fb-4c2e-8e1a-c7c368c5969b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:24.768311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46edf23-e7fb-4c2e-8e1a-c7c368c5969b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:24.768585 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46edf23-e7fb-4c2e-8e1a-c7c368c5969b None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:24.769383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46edf23-e7fb-4c2e-8e1a-c7c368c5969b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:24.769684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46edf23-e7fb-4c2e-8e1a-c7c368c5969b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:24.771167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46edf23-e7fb-4c2e-8e1a-c7c368c5969b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:24.789345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-808677e3-b5ca-4717-b902-0289b9ab02ff None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:24.790567 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-808677e3-b5ca-4717-b902-0289b9ab02ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:24.805409 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 358/733] 10.222.0.86 () {56 vars in 1135 bytes} [Fri Aug 5 19:37:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:24.806767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46edf23-e7fb-4c2e-8e1a-c7c368c5969b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:24.890728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 376/734] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:37:24.890728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808677e3-b5ca-4717-b902-0289b9ab02ff None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:24.901366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:24.956132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46edf23-e7fb-4c2e-8e1a-c7c368c5969b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:24.995634 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abd481-e10a-4d89-846b-51ef0c587509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:24.996650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abd481-e10a-4d89-846b-51ef0c587509 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:24.997136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abd481-e10a-4d89-846b-51ef0c587509 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:24.997637 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abd481-e10a-4d89-846b-51ef0c587509 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:25.010661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abd481-e10a-4d89-846b-51ef0c587509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-10abd481-e10a-4d89-846b-51ef0c587509 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:25.016456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:25.017422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:25.018252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:25.018252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:25.018252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:25.018252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:25.018252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:25.018252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:25.018252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:25.018252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:25.018252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:25.018252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:25.018252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:25.018252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:25.025977 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 359/735] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:25.027673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abd481-e10a-4d89-846b-51ef0c587509 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:25.029517 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808677e3-b5ca-4717-b902-0289b9ab02ff None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:25.051052 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:25.075805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc247968-0c1c-40c7-a8ce-1fa474760ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:25.076430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc247968-0c1c-40c7-a8ce-1fa474760ec1 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:25.076761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc247968-0c1c-40c7-a8ce-1fa474760ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:25.077032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc247968-0c1c-40c7-a8ce-1fa474760ec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:25.164044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abd481-e10a-4d89-846b-51ef0c587509 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:25.167359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc247968-0c1c-40c7-a8ce-1fa474760ec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:25.168831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc247968-0c1c-40c7-a8ce-1fa474760ec1 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:25.182704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:25.206899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb384c9-1cca-4aec-9232-d258078afe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:25.207816 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb384c9-1cca-4aec-9232-d258078afe3d None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:25.208266 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb384c9-1cca-4aec-9232-d258078afe3d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:25.208535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb384c9-1cca-4aec-9232-d258078afe3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:25.224517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb384c9-1cca-4aec-9232-d258078afe3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-0cb384c9-1cca-4aec-9232-d258078afe3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:25.230684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:25.231659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:25.232606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:25.232606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:25.232606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:25.232606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:25.232606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:25.232606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:25.232606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:25.232606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:25.232606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:25.232606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:25.232606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:37:25.232606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:25.232606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 360/736] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:37:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:25.233986 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb384c9-1cca-4aec-9232-d258078afe3d None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:25.297827 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 377/737] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:24 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:37:25.299263 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551672f1-7bb5-40aa-9362-c2692fbd83e4 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:25.299544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551672f1-7bb5-40aa-9362-c2692fbd83e4 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:25.299813 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551672f1-7bb5-40aa-9362-c2692fbd83e4 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:25.303242 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 378/738] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:25.304359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551672f1-7bb5-40aa-9362-c2692fbd83e4 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:25.315640 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:25.377159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb384c9-1cca-4aec-9232-d258078afe3d None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:25.418932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551672f1-7bb5-40aa-9362-c2692fbd83e4 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:25.423557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3caed-c39e-4aef-ab5d-c8009d553874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:25.424184 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3caed-c39e-4aef-ab5d-c8009d553874 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:25.424464 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3caed-c39e-4aef-ab5d-c8009d553874 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:25.424793 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3caed-c39e-4aef-ab5d-c8009d553874 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:25.425509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3caed-c39e-4aef-ab5d-c8009d553874 None admin] RBAC: Adding query filter params (name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:25.425812 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3caed-c39e-4aef-ab5d-c8009d553874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:25.427704 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3caed-c39e-4aef-ab5d-c8009d553874 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:25.431079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:25.441032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 361/739] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:25.441954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3caed-c39e-4aef-ab5d-c8009d553874 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:25.467275 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5cca7-fb61-4a2b-b195-528e1e02d642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:25.467900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5cca7-fb61-4a2b-b195-528e1e02d642 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:25.468179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5cca7-fb61-4a2b-b195-528e1e02d642 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:25.468454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5cca7-fb61-4a2b-b195-528e1e02d642 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:25.469247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5cca7-fb61-4a2b-b195-528e1e02d642 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:25.469554 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5cca7-fb61-4a2b-b195-528e1e02d642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:25.471038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5cca7-fb61-4a2b-b195-528e1e02d642 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:25.481586 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 379/740] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:37:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:25.483141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66332163-5690-415a-83bd-7c3cccc818dc None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:25.483327 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66332163-5690-415a-83bd-7c3cccc818dc None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:25.483607 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66332163-5690-415a-83bd-7c3cccc818dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:25.507325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-66332163-5690-415a-83bd-7c3cccc818dc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:25.509689 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-66332163-5690-415a-83bd-7c3cccc818dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:25.563904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3caed-c39e-4aef-ab5d-c8009d553874 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:25.584038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 380/741] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:37:25.585127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66332163-5690-415a-83bd-7c3cccc818dc None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:25.623072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:25.628629 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865d23d-3bd2-4037-aa9f-ea36a9d5fb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:25.629379 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865d23d-3bd2-4037-aa9f-ea36a9d5fb54 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:25.629666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865d23d-3bd2-4037-aa9f-ea36a9d5fb54 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:25.629932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865d23d-3bd2-4037-aa9f-ea36a9d5fb54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:25.641473 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865d23d-3bd2-4037-aa9f-ea36a9d5fb54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:25.642954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865d23d-3bd2-4037-aa9f-ea36a9d5fb54 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-7865d23d-3bd2-4037-aa9f-ea36a9d5fb54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:25.658661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:25.660381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:25.661958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:25.661958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:25.661958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:25.661958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:25.661958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 362/742] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:25.661958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865d23d-3bd2-4037-aa9f-ea36a9d5fb54 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:25.714658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66332163-5690-415a-83bd-7c3cccc818dc None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:25.716757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:25.762369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcefc8-3aaf-413c-a973-ca1dc137bdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:25.763309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcefc8-3aaf-413c-a973-ca1dc137bdcc None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:25.763594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcefc8-3aaf-413c-a973-ca1dc137bdcc None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:25.763867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcefc8-3aaf-413c-a973-ca1dc137bdcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:25.777871 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865d23d-3bd2-4037-aa9f-ea36a9d5fb54 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:25.779197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcefc8-3aaf-413c-a973-ca1dc137bdcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:25.782183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcefc8-3aaf-413c-a973-ca1dc137bdcc None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-1fbcefc8-3aaf-413c-a973-ca1dc137bdcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:25.807412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:25.808375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:25.809268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:25.809268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:25.809268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:25.809268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:25.809268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 381/743] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:25.809553 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcefc8-3aaf-413c-a973-ca1dc137bdcc None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:25.822899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7083f7-912d-4e0d-839a-fc38b79f9e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:25.823582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7083f7-912d-4e0d-839a-fc38b79f9e1e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:25.823862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7083f7-912d-4e0d-839a-fc38b79f9e1e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:25.824849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7083f7-912d-4e0d-839a-fc38b79f9e1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:25.837001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7083f7-912d-4e0d-839a-fc38b79f9e1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:25.838222 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7083f7-912d-4e0d-839a-fc38b79f9e1e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-1b7083f7-912d-4e0d-839a-fc38b79f9e1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:25.855191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:25.856297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:25.857207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:25.857207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:25.857207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:25.857207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:25.857207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 363/744] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:25.857967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7083f7-912d-4e0d-839a-fc38b79f9e1e None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:25.898545 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:25.934000 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcefc8-3aaf-413c-a973-ca1dc137bdcc None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:25.970962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ebd82-dc4a-4234-b95e-c47f0f667c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:25.971493 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ebd82-dc4a-4234-b95e-c47f0f667c6f None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:25.971768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ebd82-dc4a-4234-b95e-c47f0f667c6f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:25.972041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ebd82-dc4a-4234-b95e-c47f0f667c6f None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:25.972729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ebd82-dc4a-4234-b95e-c47f0f667c6f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:25.974089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ebd82-dc4a-4234-b95e-c47f0f667c6f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:25.975057 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7083f7-912d-4e0d-839a-fc38b79f9e1e None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:25.994664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 382/745] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:37:25 2022] POST /identity/v3/services => generated 240 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:37:25.994664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ebd82-dc4a-4234-b95e-c47f0f667c6f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:26.028227 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:26.029251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da74b12-8f8f-4986-be0f-64f9357022f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:26.030414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da74b12-8f8f-4986-be0f-64f9357022f2 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:26.031092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da74b12-8f8f-4986-be0f-64f9357022f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:26.031712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da74b12-8f8f-4986-be0f-64f9357022f2 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:26.033045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da74b12-8f8f-4986-be0f-64f9357022f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:26.033708 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da74b12-8f8f-4986-be0f-64f9357022f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:26.036218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da74b12-8f8f-4986-be0f-64f9357022f2 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:26.056521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 364/746] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:26.057512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da74b12-8f8f-4986-be0f-64f9357022f2 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:26.166683 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ebd82-dc4a-4234-b95e-c47f0f667c6f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:26.174446 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:26.186082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da74b12-8f8f-4986-be0f-64f9357022f2 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:26.211024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20651fbb-328d-4acd-b7c2-dcae85d819e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:26.212489 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20651fbb-328d-4acd-b7c2-dcae85d819e1 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:26.212931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20651fbb-328d-4acd-b7c2-dcae85d819e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:26.213430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20651fbb-328d-4acd-b7c2-dcae85d819e1 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:26.214812 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20651fbb-328d-4acd-b7c2-dcae85d819e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:26.215369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20651fbb-328d-4acd-b7c2-dcae85d819e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:26.216916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20651fbb-328d-4acd-b7c2-dcae85d819e1 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:26.231569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36524c59-c1ca-4c7d-8816-dcd9f8a5f148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:26.232118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36524c59-c1ca-4c7d-8816-dcd9f8a5f148 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:26.232383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36524c59-c1ca-4c7d-8816-dcd9f8a5f148 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:26.232741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36524c59-c1ca-4c7d-8816-dcd9f8a5f148 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:26.233632 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36524c59-c1ca-4c7d-8816-dcd9f8a5f148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:26.233883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 383/747] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:26.233989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36524c59-c1ca-4c7d-8816-dcd9f8a5f148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:26.235015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20651fbb-328d-4acd-b7c2-dcae85d819e1 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:26.235593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36524c59-c1ca-4c7d-8816-dcd9f8a5f148 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:26.251360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 365/748] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:26.252448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36524c59-c1ca-4c7d-8816-dcd9f8a5f148 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:26.347012 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:26.371284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20651fbb-328d-4acd-b7c2-dcae85d819e1 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:26.376589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36524c59-c1ca-4c7d-8816-dcd9f8a5f148 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:26.411296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68c630-2fc0-42bb-8a8a-88c6b69091bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:26.412330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68c630-2fc0-42bb-8a8a-88c6b69091bf None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:26.412916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68c630-2fc0-42bb-8a8a-88c6b69091bf None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:26.413413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68c630-2fc0-42bb-8a8a-88c6b69091bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:26.415953 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ac282-75c9-423e-a2c4-a8ec0f7f69c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:26.416972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ac282-75c9-423e-a2c4-a8ec0f7f69c6 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:26.416972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ac282-75c9-423e-a2c4-a8ec0f7f69c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:26.417399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ac282-75c9-423e-a2c4-a8ec0f7f69c6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:26.422150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68c630-2fc0-42bb-8a8a-88c6b69091bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:26.423873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68c630-2fc0-42bb-8a8a-88c6b69091bf None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:26.426556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ac282-75c9-423e-a2c4-a8ec0f7f69c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:26.428278 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ac282-75c9-423e-a2c4-a8ec0f7f69c6 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-bc68c630-2fc0-42bb-8a8a-88c6b69091bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:26.437817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:26.439084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:26.439987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:26.439987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:26.439987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:26.439987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:26.439987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:26.439987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:26.439987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:26.439987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:26.442370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 384/749] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:26.444217 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68c630-2fc0-42bb-8a8a-88c6b69091bf None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-d84ac282-75c9-423e-a2c4-a8ec0f7f69c6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:26.446172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:26.447510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:26.448706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:26.448706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:26.448706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:26.448706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:26.448706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:26.448706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:26.448706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:26.448706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:26.448706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:37:26.448706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:26.449322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 366/750] 10.222.0.86 () {56 vars in 1117 bytes} [Fri Aug 5 19:37:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:26.454351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ac282-75c9-423e-a2c4-a8ec0f7f69c6 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:26.471649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:26.573609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ac282-75c9-423e-a2c4-a8ec0f7f69c6 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:26.579741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68c630-2fc0-42bb-8a8a-88c6b69091bf None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:26.622142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06626f-27b9-40b0-8200-5b0e649bcc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:26.622912 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06626f-27b9-40b0-8200-5b0e649bcc72 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:26.623279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06626f-27b9-40b0-8200-5b0e649bcc72 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:26.623621 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06626f-27b9-40b0-8200-5b0e649bcc72 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:26.624658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06626f-27b9-40b0-8200-5b0e649bcc72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:26.624831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06626f-27b9-40b0-8200-5b0e649bcc72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:26.626248 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06626f-27b9-40b0-8200-5b0e649bcc72 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:26.632627 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401178-1234-487c-be5a-59e12dd81d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:26.633558 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401178-1234-487c-be5a-59e12dd81d27 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:26.634076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401178-1234-487c-be5a-59e12dd81d27 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:26.634559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca401178-1234-487c-be5a-59e12dd81d27 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:26.639219 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:26.643103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 367/751] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:26.645002 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06626f-27b9-40b0-8200-5b0e649bcc72 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:26.651566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca401178-1234-487c-be5a-59e12dd81d27 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:26.652488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca401178-1234-487c-be5a-59e12dd81d27 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-ca401178-1234-487c-be5a-59e12dd81d27 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:26.665535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:26.667362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:26.669116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:26.669116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:26.669116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:26.669116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:26.669116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:26.669116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:26.669116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:26.669116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:26.669116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:37:26.669116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:26.669116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 385/752] 10.222.0.86 () {56 vars in 1093 bytes} [Fri Aug 5 19:37:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:26.669116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401178-1234-487c-be5a-59e12dd81d27 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:26.770880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:26.788288 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca401178-1234-487c-be5a-59e12dd81d27 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:26.818319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06626f-27b9-40b0-8200-5b0e649bcc72 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:26.830234 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb053e-8cfc-436d-a6b5-5761d35b705f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:26.830780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb053e-8cfc-436d-a6b5-5761d35b705f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:26.831238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb053e-8cfc-436d-a6b5-5761d35b705f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:26.831370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb053e-8cfc-436d-a6b5-5761d35b705f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:26.832158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb053e-8cfc-436d-a6b5-5761d35b705f None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=e8adb0fd1fbc4268b70f3936036e550b) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:26.832464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb053e-8cfc-436d-a6b5-5761d35b705f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:26.834415 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb053e-8cfc-436d-a6b5-5761d35b705f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:26.846887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 386/753] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:26 2022] GET /identity/v3/role_assignments?user.id=e8adb0fd1fbc4268b70f3936036e550b&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=93eb204498fc49d297bcd1841e00818d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:26.846887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb053e-8cfc-436d-a6b5-5761d35b705f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:26.870930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75daa49-8db1-4fe6-b298-fa78943597e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:26.872004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75daa49-8db1-4fe6-b298-fa78943597e5 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:26.872482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75daa49-8db1-4fe6-b298-fa78943597e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:26.873190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75daa49-8db1-4fe6-b298-fa78943597e5 None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:26.874448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75daa49-8db1-4fe6-b298-fa78943597e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:26.874803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75daa49-8db1-4fe6-b298-fa78943597e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:26.876450 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75daa49-8db1-4fe6-b298-fa78943597e5 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:26.904967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 368/754] 10.222.0.86 () {56 vars in 1149 bytes} [Fri Aug 5 19:37:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:26.906585 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c5df4-64f4-4979-a9f4-44ba023ca3fd None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:26.906892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c5df4-64f4-4979-a9f4-44ba023ca3fd None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:26.907162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c5df4-64f4-4979-a9f4-44ba023ca3fd None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:26.908843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 369/755] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:26.909663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c5df4-64f4-4979-a9f4-44ba023ca3fd None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:26.928362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:26.987009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb053e-8cfc-436d-a6b5-5761d35b705f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:27.034799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4b4d1-a140-4a64-bc69-f90af7f2f045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:27.035342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4b4d1-a140-4a64-bc69-f90af7f2f045 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:27.035617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4b4d1-a140-4a64-bc69-f90af7f2f045 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:27.035889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4b4d1-a140-4a64-bc69-f90af7f2f045 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:27.036713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4b4d1-a140-4a64-bc69-f90af7f2f045 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:27.037020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4b4d1-a140-4a64-bc69-f90af7f2f045 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:27.039334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4b4d1-a140-4a64-bc69-f90af7f2f045 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:27.061068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 387/756] 10.222.0.86 () {56 vars in 1133 bytes} [Fri Aug 5 19:37:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:27.062400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d16d3-91da-41dc-b1a6-6d518e11677b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:27.062707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d16d3-91da-41dc-b1a6-6d518e11677b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:27.063235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d16d3-91da-41dc-b1a6-6d518e11677b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:27.084097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c00d16d3-91da-41dc-b1a6-6d518e11677b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:27.086271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c00d16d3-91da-41dc-b1a6-6d518e11677b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:27.171005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 388/757] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:27.172257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d16d3-91da-41dc-b1a6-6d518e11677b None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:27.184440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c5df4-64f4-4979-a9f4-44ba023ca3fd None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:27.222792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:27.231982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ac1cc-b5e1-42e5-8f86-36d5ebaa1f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:27.232829 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:27.233874 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ac1cc-b5e1-42e5-8f86-36d5ebaa1f75 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:27.234419 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ac1cc-b5e1-42e5-8f86-36d5ebaa1f75 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:27.235043 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ac1cc-b5e1-42e5-8f86-36d5ebaa1f75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:27.302306 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d16d3-91da-41dc-b1a6-6d518e11677b None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:27.315978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ac1cc-b5e1-42e5-8f86-36d5ebaa1f75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:27.317786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ac1cc-b5e1-42e5-8f86-36d5ebaa1f75 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:27.342515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfde49-b418-42bb-9ddb-8a35153d86c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:27.343116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfde49-b418-42bb-9ddb-8a35153d86c0 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:27.343395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfde49-b418-42bb-9ddb-8a35153d86c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:27.343664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfde49-b418-42bb-9ddb-8a35153d86c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:27.356552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfde49-b418-42bb-9ddb-8a35153d86c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:27.358030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdfde49-b418-42bb-9ddb-8a35153d86c0 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-afdfde49-b418-42bb-9ddb-8a35153d86c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:27.379110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:27.380105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:27.381059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:27.381059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:27.381059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:27.381059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:27.381059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 389/758] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:27.381810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfde49-b418-42bb-9ddb-8a35153d86c0 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:27.436191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 370/759] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:37:27.437408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ac1cc-b5e1-42e5-8f86-36d5ebaa1f75 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:27.470215 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:27.517988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:27.524797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfde49-b418-42bb-9ddb-8a35153d86c0 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:27.555346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ac1cc-b5e1-42e5-8f86-36d5ebaa1f75 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:27.567422 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256bce9-5534-4673-be54-e9385c0365d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:27.567984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256bce9-5534-4673-be54-e9385c0365d0 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:27.568297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256bce9-5534-4673-be54-e9385c0365d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:27.568633 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256bce9-5534-4673-be54-e9385c0365d0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:27.569382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256bce9-5534-4673-be54-e9385c0365d0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:27.570658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256bce9-5534-4673-be54-e9385c0365d0 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-f256bce9-5534-4673-be54-e9385c0365d0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:27.580474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:27.581442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:37:27.582272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:27.587067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 390/760] 10.222.0.86 () {56 vars in 1129 bytes} [Fri Aug 5 19:37:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:27.587944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256bce9-5534-4673-be54-e9385c0365d0 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:27.597461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4698ad-8779-41d6-90f1-beaf7fc7b44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:27.598357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4698ad-8779-41d6-90f1-beaf7fc7b44a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:27.598907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4698ad-8779-41d6-90f1-beaf7fc7b44a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:27.599395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4698ad-8779-41d6-90f1-beaf7fc7b44a None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:27.600892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4698ad-8779-41d6-90f1-beaf7fc7b44a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:27.601456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4698ad-8779-41d6-90f1-beaf7fc7b44a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:27.603195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4698ad-8779-41d6-90f1-beaf7fc7b44a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:27.616642 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 371/761] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:27.617825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4698ad-8779-41d6-90f1-beaf7fc7b44a None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:27.732674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4698ad-8779-41d6-90f1-beaf7fc7b44a None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:27.737202 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256bce9-5534-4673-be54-e9385c0365d0 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:27.755439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:27.777117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cac91-f7e0-49ac-bac0-c033b0e1d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:27.778260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cac91-f7e0-49ac-bac0-c033b0e1d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:27.779077 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cac91-f7e0-49ac-bac0-c033b0e1d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:27.779680 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cac91-f7e0-49ac-bac0-c033b0e1d9a9 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:27.780566 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225cac91-f7e0-49ac-bac0-c033b0e1d9a9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:27.781062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225cac91-f7e0-49ac-bac0-c033b0e1d9a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:27.782653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225cac91-f7e0-49ac-bac0-c033b0e1d9a9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:27.783324 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ec731-d175-409f-86ce-184904240dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:27.784063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ec731-d175-409f-86ce-184904240dc8 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:27.784691 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ec731-d175-409f-86ce-184904240dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:27.785041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ec731-d175-409f-86ce-184904240dc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:27.790385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 372/762] 10.222.0.86 () {56 vars in 1124 bytes} [Fri Aug 5 19:37:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:27.791488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cac91-f7e0-49ac-bac0-c033b0e1d9a9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:27.799248 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ec731-d175-409f-86ce-184904240dc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:27.800943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ec731-d175-409f-86ce-184904240dc8 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-d12ec731-d175-409f-86ce-184904240dc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:27.823531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:27.824645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:27.825587 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:27.825587 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:27.825587 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:27.825587 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:27.825587 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 391/763] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:27.826754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ec731-d175-409f-86ce-184904240dc8 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:27.849058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:27.915849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cac91-f7e0-49ac-bac0-c033b0e1d9a9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:27.956359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cee42f-a7ae-4ce4-b654-b90c10ec3c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:27.957297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cee42f-a7ae-4ce4-b654-b90c10ec3c9e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:27.957802 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cee42f-a7ae-4ce4-b654-b90c10ec3c9e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:27.958270 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cee42f-a7ae-4ce4-b654-b90c10ec3c9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:27.967223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cee42f-a7ae-4ce4-b654-b90c10ec3c9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:27.969137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cee42f-a7ae-4ce4-b654-b90c10ec3c9e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:27.970877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ec731-d175-409f-86ce-184904240dc8 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-25cee42f-a7ae-4ce4-b654-b90c10ec3c9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:27.982906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:27.983855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:27.984725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:27.984725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:27.984725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:27.984725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:27.984725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:27.984725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:27.984725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:27.984725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:27.987088 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 373/764] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:27.988129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cee42f-a7ae-4ce4-b654-b90c10ec3c9e None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:28.008704 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:28.021832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a9386-ea75-476c-8b10-b9506860238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:28.022744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9386-ea75-476c-8b10-b9506860238e None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:28.023088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9386-ea75-476c-8b10-b9506860238e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:28.023360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a9386-ea75-476c-8b10-b9506860238e None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:28.024327 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a9386-ea75-476c-8b10-b9506860238e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:28.024630 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a9386-ea75-476c-8b10-b9506860238e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:28.026284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a9386-ea75-476c-8b10-b9506860238e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:28.034099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 392/765] 10.222.0.86 () {56 vars in 1124 bytes} [Fri Aug 5 19:37:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:28.035297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a9386-ea75-476c-8b10-b9506860238e None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:28.130019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cee42f-a7ae-4ce4-b654-b90c10ec3c9e None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:28.136511 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:28.168621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a9386-ea75-476c-8b10-b9506860238e None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:28.180837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730412b6-a9e0-4d1c-ab2d-9385cb7c87d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:28.181459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730412b6-a9e0-4d1c-ab2d-9385cb7c87d0 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:28.181736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730412b6-a9e0-4d1c-ab2d-9385cb7c87d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:28.182005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730412b6-a9e0-4d1c-ab2d-9385cb7c87d0 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:28.182947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730412b6-a9e0-4d1c-ab2d-9385cb7c87d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:28.183063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730412b6-a9e0-4d1c-ab2d-9385cb7c87d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:28.184663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730412b6-a9e0-4d1c-ab2d-9385cb7c87d0 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:28.198588 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 374/766] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:28.199513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730412b6-a9e0-4d1c-ab2d-9385cb7c87d0 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:28.217334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac658a1-f38a-41bb-bdde-3ae863d130bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:28.218534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac658a1-f38a-41bb-bdde-3ae863d130bd None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:28.218986 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac658a1-f38a-41bb-bdde-3ae863d130bd None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:28.219671 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac658a1-f38a-41bb-bdde-3ae863d130bd None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:28.221097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac658a1-f38a-41bb-bdde-3ae863d130bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:28.221818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac658a1-f38a-41bb-bdde-3ae863d130bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:28.223719 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac658a1-f38a-41bb-bdde-3ae863d130bd None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:28.242689 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 393/767] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:28.245775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac658a1-f38a-41bb-bdde-3ae863d130bd None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:28.310290 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:28.346484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730412b6-a9e0-4d1c-ab2d-9385cb7c87d0 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:28.365988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac658a1-f38a-41bb-bdde-3ae863d130bd None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:28.388572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ebb98-39ed-4c23-8566-8e5deae981b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:28.389201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ebb98-39ed-4c23-8566-8e5deae981b3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:28.389546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ebb98-39ed-4c23-8566-8e5deae981b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:28.390131 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ebb98-39ed-4c23-8566-8e5deae981b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:28.390818 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ebb98-39ed-4c23-8566-8e5deae981b3 None admin] RBAC: Adding query filter params (interface=public, service_id=b64bcb9f7d7d43dfa59dcdf76ea25094, region_id=RegionOne) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:28.391288 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ebb98-39ed-4c23-8566-8e5deae981b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:28.393890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ebb98-39ed-4c23-8566-8e5deae981b3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:28.409367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 375/768] 10.222.0.86 () {56 vars in 1251 bytes} [Fri Aug 5 19:37:28 2022] GET /identity/v3/endpoints?service_id=b64bcb9f7d7d43dfa59dcdf76ea25094&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:28.409367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ebb98-39ed-4c23-8566-8e5deae981b3 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:28.413470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480046c5-55e4-4a1b-8888-303282903657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:28.414135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480046c5-55e4-4a1b-8888-303282903657 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:28.414411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480046c5-55e4-4a1b-8888-303282903657 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:28.414711 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480046c5-55e4-4a1b-8888-303282903657 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:28.422034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480046c5-55e4-4a1b-8888-303282903657 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:28.423528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480046c5-55e4-4a1b-8888-303282903657 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:28.425400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-480046c5-55e4-4a1b-8888-303282903657 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:28.436954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:28.438200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:28.439145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:28.439145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:28.439145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:28.439145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:28.439145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:28.439145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:28.439145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:28.439145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:28.439145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 394/769] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:28.439979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51ce23-c31d-4602-9b72-75376e89c42d None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:28.439979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51ce23-c31d-4602-9b72-75376e89c42d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:28.440238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51ce23-c31d-4602-9b72-75376e89c42d None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:28.441665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 395/770] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:28.442329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51ce23-c31d-4602-9b72-75376e89c42d None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:28.548375 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ebb98-39ed-4c23-8566-8e5deae981b3 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:28.573622 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51ce23-c31d-4602-9b72-75376e89c42d None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:28.591581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23360f-def5-48da-af56-0c5a2717f9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:28.592380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23360f-def5-48da-af56-0c5a2717f9db None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:28.592663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23360f-def5-48da-af56-0c5a2717f9db None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:28.592944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23360f-def5-48da-af56-0c5a2717f9db None admin] PATH_INFO: `/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/ab8f9d9cf4ca4e38ae6eb67e949b40fd/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:28.594986 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23360f-def5-48da-af56-0c5a2717f9db None admin] RBAC: Authorizing `identity:create_grant(project_id=9c73d2cfaa1c47829ca6936186d2bb26, user_id=ab8f9d9cf4ca4e38ae6eb67e949b40fd, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:28.596418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23360f-def5-48da-af56-0c5a2717f9db None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:28.608861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:28.618760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 376/771] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:37:28 2022] PUT /identity/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/ab8f9d9cf4ca4e38ae6eb67e949b40fd/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:37:28.618760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23360f-def5-48da-af56-0c5a2717f9db None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:28.627114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a604894-20e3-4420-b324-d134c54da591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:28.628061 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a604894-20e3-4420-b324-d134c54da591 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:28.628061 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a604894-20e3-4420-b324-d134c54da591 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:28.628337 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a604894-20e3-4420-b324-d134c54da591 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:28.629023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a604894-20e3-4420-b324-d134c54da591 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:28.630430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a604894-20e3-4420-b324-d134c54da591 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:28.655471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 396/772] 10.222.0.86 () {56 vars in 1087 bytes} [Fri Aug 5 19:37:28 2022] GET /identity/v3/services => generated 772 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:28.656917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117f6ae-06eb-4a97-b153-545a06cbae56 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:28.657204 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117f6ae-06eb-4a97-b153-545a06cbae56 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:28.657477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117f6ae-06eb-4a97-b153-545a06cbae56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:28.686550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-f117f6ae-06eb-4a97-b153-545a06cbae56 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:28.688494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f117f6ae-06eb-4a97-b153-545a06cbae56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:28.837814 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:28.861408 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 397/773] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:28.862031 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23360f-def5-48da-af56-0c5a2717f9db None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:28.885888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117f6ae-06eb-4a97-b153-545a06cbae56 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:28.914096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a12d1-e825-4172-bc37-1fe6fa33a62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:28.914673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a12d1-e825-4172-bc37-1fe6fa33a62a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:28.914964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a12d1-e825-4172-bc37-1fe6fa33a62a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:28.915231 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a12d1-e825-4172-bc37-1fe6fa33a62a None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:28.915996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a12d1-e825-4172-bc37-1fe6fa33a62a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:28.916306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a12d1-e825-4172-bc37-1fe6fa33a62a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:28.917751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a12d1-e825-4172-bc37-1fe6fa33a62a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:28.936450 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 377/774] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:28.937830 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a12d1-e825-4172-bc37-1fe6fa33a62a None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:29.003237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:29.019153 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117f6ae-06eb-4a97-b153-545a06cbae56 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:29.049603 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a12d1-e825-4172-bc37-1fe6fa33a62a None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:29.077072 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8505f-e456-4acf-b49e-686463cccb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:29.077873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8505f-e456-4acf-b49e-686463cccb8b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:29.078230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8505f-e456-4acf-b49e-686463cccb8b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:29.078650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8505f-e456-4acf-b49e-686463cccb8b None admin] PATH_INFO: `/v3/services/network` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:29.079542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8505f-e456-4acf-b49e-686463cccb8b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:29.081130 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8505f-e456-4acf-b49e-686463cccb8b None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:29.088903 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef443ab-2f96-451a-bc77-8e4af6306501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:29.089528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef443ab-2f96-451a-bc77-8e4af6306501 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:29.089793 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef443ab-2f96-451a-bc77-8e4af6306501 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:29.090069 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef443ab-2f96-451a-bc77-8e4af6306501 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:29.090895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef443ab-2f96-451a-bc77-8e4af6306501 None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=7fc4868f25364311a504224fcd2c5a73) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:29.091201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef443ab-2f96-451a-bc77-8e4af6306501 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:29.092638 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef443ab-2f96-451a-bc77-8e4af6306501 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-31d8505f-e456-4acf-b49e-686463cccb8b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:29.092942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:29.094100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:37:29.095152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:29.098690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 398/775] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:28 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:29.098690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28033-5144-4bcc-aa37-bd5f4f78087c None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:29.098690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28033-5144-4bcc-aa37-bd5f4f78087c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:29.099075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce28033-5144-4bcc-aa37-bd5f4f78087c None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:29.101360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 399/776] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:29.104837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 378/777] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:28 2022] GET /identity/v3/role_assignments?user.id=7fc4868f25364311a504224fcd2c5a73&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:29.106841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd10d9-2d44-4b38-bf85-fc69d0cf6e9c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:29.107267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd10d9-2d44-4b38-bf85-fc69d0cf6e9c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:29.107673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd10d9-2d44-4b38-bf85-fc69d0cf6e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:29.113555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef443ab-2f96-451a-bc77-8e4af6306501 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:29.145070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-12cd10d9-2d44-4b38-bf85-fc69d0cf6e9c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:29.147769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-12cd10d9-2d44-4b38-bf85-fc69d0cf6e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:29.245819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef443ab-2f96-451a-bc77-8e4af6306501 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:29.280867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:29.293894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:29.315987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 400/778] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:29.324597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9b26e-04c6-465e-9c6f-d15e1517786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:29.325266 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9b26e-04c6-465e-9c6f-d15e1517786b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:29.325503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9b26e-04c6-465e-9c6f-d15e1517786b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:29.325772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9b26e-04c6-465e-9c6f-d15e1517786b None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:29.326465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9b26e-04c6-465e-9c6f-d15e1517786b None admin] RBAC: Adding query filter params (name=network) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:29.326805 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9b26e-04c6-465e-9c6f-d15e1517786b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:29.328201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9b26e-04c6-465e-9c6f-d15e1517786b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:29.338105 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 379/779] 10.222.0.86 () {56 vars in 1112 bytes} [Fri Aug 5 19:37:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:29.343970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd10d9-2d44-4b38-bf85-fc69d0cf6e9c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:29.351510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9b26e-04c6-465e-9c6f-d15e1517786b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:29.468500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd10d9-2d44-4b38-bf85-fc69d0cf6e9c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:29.514272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a4d0-4f1e-46fa-965d-43ce8d789931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:29.515223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a4d0-4f1e-46fa-965d-43ce8d789931 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:29.515651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a4d0-4f1e-46fa-965d-43ce8d789931 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:29.517067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a4d0-4f1e-46fa-965d-43ce8d789931 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:29.517940 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a4d0-4f1e-46fa-965d-43ce8d789931 None admin] RBAC: Adding query filter params (type=network) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:29.519367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a4d0-4f1e-46fa-965d-43ce8d789931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:29.520786 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a4d0-4f1e-46fa-965d-43ce8d789931 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:29.536287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 401/780] 10.222.0.86 () {56 vars in 1112 bytes} [Fri Aug 5 19:37:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:29.538044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a4d0-4f1e-46fa-965d-43ce8d789931 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:29.550978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9b26e-04c6-465e-9c6f-d15e1517786b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:29.618608 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dc064-7517-43f3-9845-af1d6681ece9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:29.619640 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dc064-7517-43f3-9845-af1d6681ece9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:29.620146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dc064-7517-43f3-9845-af1d6681ece9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:29.622215 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dc064-7517-43f3-9845-af1d6681ece9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:29.633765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:29.645978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65dc064-7517-43f3-9845-af1d6681ece9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-b65dc064-7517-43f3-9845-af1d6681ece9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:29.652674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:29.653648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:29.654716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:29.654716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:29.654716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:29.654716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:29.654716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:29.654716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:29.654716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:29.654716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:29.654716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:29.654716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:29.654716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:37:29.654716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:29.654716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 380/781] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:37:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:29.661366 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dc064-7517-43f3-9845-af1d6681ece9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:29.751723 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a4d0-4f1e-46fa-965d-43ce8d789931 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:29.778885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:29.837211 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cd82d-0266-4301-9366-248bafa418ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:29.837848 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cd82d-0266-4301-9366-248bafa418ab None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:29.838123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cd82d-0266-4301-9366-248bafa418ab None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:29.838560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cd82d-0266-4301-9366-248bafa418ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:29.839927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cd82d-0266-4301-9366-248bafa418ab None admin] RBAC: Adding query filter params (interface=public, service_id=5e110b99b99e43e8af665d06a44d181a, region_id=RegionOne) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:29.840461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cd82d-0266-4301-9366-248bafa418ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:29.844655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cd82d-0266-4301-9366-248bafa418ab None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:29.867572 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 402/782] 10.222.0.86 () {56 vars in 1251 bytes} [Fri Aug 5 19:37:29 2022] GET /identity/v3/endpoints?service_id=5e110b99b99e43e8af665d06a44d181a&interface=public®ion_id=RegionOne => generated 194 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:29.873938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cd82d-0266-4301-9366-248bafa418ab None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:29.882969 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dc064-7517-43f3-9845-af1d6681ece9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:29.954022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65275ec6-bbab-44d6-8bb8-cf2c0becfad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:29.954660 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65275ec6-bbab-44d6-8bb8-cf2c0becfad4 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:29.954941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65275ec6-bbab-44d6-8bb8-cf2c0becfad4 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:29.955209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65275ec6-bbab-44d6-8bb8-cf2c0becfad4 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:29.955923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65275ec6-bbab-44d6-8bb8-cf2c0becfad4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:29.956219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65275ec6-bbab-44d6-8bb8-cf2c0becfad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:29.960387 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65275ec6-bbab-44d6-8bb8-cf2c0becfad4 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:29.978279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 381/783] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:37:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:29.987509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65275ec6-bbab-44d6-8bb8-cf2c0becfad4 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:30.084805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cd82d-0266-4301-9366-248bafa418ab None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:30.114116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:30.163116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8535e-fc42-4efa-9672-5152c264933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:30.163954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8535e-fc42-4efa-9672-5152c264933a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:30.164230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8535e-fc42-4efa-9672-5152c264933a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:30.164499 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8535e-fc42-4efa-9672-5152c264933a None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:30.165456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8535e-fc42-4efa-9672-5152c264933a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:30.167904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d8535e-fc42-4efa-9672-5152c264933a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:30.187153 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 403/784] 10.222.0.86 () {56 vars in 1087 bytes} [Fri Aug 5 19:37:29 2022] GET /identity/v3/services => generated 772 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:30.201855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65275ec6-bbab-44d6-8bb8-cf2c0becfad4 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:30.227150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:30.261413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b53b82-3ed6-47cd-a648-de3a002e0f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:30.262051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b53b82-3ed6-47cd-a648-de3a002e0f0a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:30.262326 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b53b82-3ed6-47cd-a648-de3a002e0f0a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:30.262594 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b53b82-3ed6-47cd-a648-de3a002e0f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:30.282826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b53b82-3ed6-47cd-a648-de3a002e0f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:30.284278 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b53b82-3ed6-47cd-a648-de3a002e0f0a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-28b53b82-3ed6-47cd-a648-de3a002e0f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:30.304353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:30.305500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:30.306658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:30.306658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:30.306658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:30.306658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:30.306658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 382/785] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:30.313480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8535e-fc42-4efa-9672-5152c264933a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:30.470995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8535e-fc42-4efa-9672-5152c264933a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:30.539414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0b460-5c45-47c2-b189-ceb837b3c72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:30.540055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0b460-5c45-47c2-b189-ceb837b3c72a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:30.540330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0b460-5c45-47c2-b189-ceb837b3c72a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:30.540601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0b460-5c45-47c2-b189-ceb837b3c72a None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:30.541308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0b460-5c45-47c2-b189-ceb837b3c72a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:30.541609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0b460-5c45-47c2-b189-ceb837b3c72a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:30.543088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0b460-5c45-47c2-b189-ceb837b3c72a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:30.559239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 404/786] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:30.565105 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b53b82-3ed6-47cd-a648-de3a002e0f0a None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:30.683106 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b53b82-3ed6-47cd-a648-de3a002e0f0a None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:30.722272 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec364b-740d-4f55-9280-fe83f0cadc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:30.722933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec364b-740d-4f55-9280-fe83f0cadc0a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:30.723208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec364b-740d-4f55-9280-fe83f0cadc0a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:30.723472 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec364b-740d-4f55-9280-fe83f0cadc0a None admin] PATH_INFO: `/v3/users/swift` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:30.733408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec364b-740d-4f55-9280-fe83f0cadc0a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:30.734869 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec364b-740d-4f55-9280-fe83f0cadc0a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-78ec364b-740d-4f55-9280-fe83f0cadc0a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:30.748494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:30.749517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:30.750466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:30.750466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:30.750466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:30.750466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:30.750466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:30.750466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:30.750466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:30.750466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:30.750466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:37:30.750466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:30.750466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 383/787] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:37:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:30.756573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0b460-5c45-47c2-b189-ceb837b3c72a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:30.829635 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:30.923930 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0b460-5c45-47c2-b189-ceb837b3c72a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:30.980903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80eff3-0b8f-4687-b291-0515bde88977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:30.981528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80eff3-0b8f-4687-b291-0515bde88977 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:30.981830 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80eff3-0b8f-4687-b291-0515bde88977 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:30.982104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80eff3-0b8f-4687-b291-0515bde88977 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:30.982940 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80eff3-0b8f-4687-b291-0515bde88977 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:30.983248 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80eff3-0b8f-4687-b291-0515bde88977 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:30.984694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80eff3-0b8f-4687-b291-0515bde88977 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:31.012619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 405/788] 10.222.0.86 () {56 vars in 1135 bytes} [Fri Aug 5 19:37:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:31.018305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec364b-740d-4f55-9280-fe83f0cadc0a None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:31.060917 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:31.174230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec364b-740d-4f55-9280-fe83f0cadc0a None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:31.231529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f585f5-53e0-446f-be50-575955b8d584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:31.232111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f585f5-53e0-446f-be50-575955b8d584 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:31.232393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f585f5-53e0-446f-be50-575955b8d584 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:31.232663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f585f5-53e0-446f-be50-575955b8d584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:31.248966 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f585f5-53e0-446f-be50-575955b8d584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:31.250434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f585f5-53e0-446f-be50-575955b8d584 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-f1f585f5-53e0-446f-be50-575955b8d584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:31.267889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:31.268927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:31.269810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:31.269810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:31.269810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:31.269810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:31.269810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 384/789] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:31.279788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80eff3-0b8f-4687-b291-0515bde88977 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:31.335537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbfb17-c6db-4d6c-81ab-b8e93fee265d None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:31.335907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbfb17-c6db-4d6c-81ab-b8e93fee265d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:31.336086 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbfb17-c6db-4d6c-81ab-b8e93fee265d None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:31.337776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 385/790] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:31.342866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1fbe3-e626-4d08-a50f-ceebc490b692 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:31.343153 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1fbe3-e626-4d08-a50f-ceebc490b692 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:31.343430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1fbe3-e626-4d08-a50f-ceebc490b692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:31.368221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-61b1fbe3-e626-4d08-a50f-ceebc490b692 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:31.369148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-61b1fbe3-e626-4d08-a50f-ceebc490b692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:31.407069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:31.416948 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80eff3-0b8f-4687-b291-0515bde88977 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:31.448624 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 386/791] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:37:31.450280 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1b828-be6f-4b31-9399-89da4b38d0a7 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:31.450551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1b828-be6f-4b31-9399-89da4b38d0a7 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:31.450887 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1b828-be6f-4b31-9399-89da4b38d0a7 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:31.456346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 387/792] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:31.457715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bc7e1-d560-4e88-a353-ffc7740b6b98 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:31.457994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bc7e1-d560-4e88-a353-ffc7740b6b98 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:31.458259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bc7e1-d560-4e88-a353-ffc7740b6b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:31.466345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:31.471144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc91b2-210a-4ac9-9807-d298eee97e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:31.472137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc91b2-210a-4ac9-9807-d298eee97e4a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:31.472619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc91b2-210a-4ac9-9807-d298eee97e4a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:31.473152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc91b2-210a-4ac9-9807-d298eee97e4a None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:31.474760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc91b2-210a-4ac9-9807-d298eee97e4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:31.475304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc91b2-210a-4ac9-9807-d298eee97e4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:31.477307 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc91b2-210a-4ac9-9807-d298eee97e4a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:31.481999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-b39bc7e1-d560-4e88-a353-ffc7740b6b98 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:31.483210 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b39bc7e1-d560-4e88-a353-ffc7740b6b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:31.492523 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 406/793] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:31.498516 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc91b2-210a-4ac9-9807-d298eee97e4a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:31.578648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 388/794] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:37:31.578648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bc7e1-d560-4e88-a353-ffc7740b6b98 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:31.629968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc91b2-210a-4ac9-9807-d298eee97e4a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:31.680592 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80147582-7b11-4cf4-ab6b-af99f012c042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:31.681512 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80147582-7b11-4cf4-ab6b-af99f012c042 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:31.681916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80147582-7b11-4cf4-ab6b-af99f012c042 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:31.682321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80147582-7b11-4cf4-ab6b-af99f012c042 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:31.695454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bc7e1-d560-4e88-a353-ffc7740b6b98 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:31.698667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80147582-7b11-4cf4-ab6b-af99f012c042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-80147582-7b11-4cf4-ab6b-af99f012c042 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:31.704781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:31.705775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:31.706699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:31.706699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:31.706699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:31.706699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:31.706699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:31.706699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:31.706699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:31.706699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:31.706699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:31.706699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:31.706699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:37:31.706699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:31.707376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 407/795] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:37:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:31.712107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb55a0-ed81-4572-837e-d51cd20b31f3 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:31.712107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb55a0-ed81-4572-837e-d51cd20b31f3 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:31.712107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb55a0-ed81-4572-837e-d51cd20b31f3 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:31.712107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 408/796] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:31.712921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb55a0-ed81-4572-837e-d51cd20b31f3 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:31.734121 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:31.738341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:31.743251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd0c48-f9d6-482b-adce-306a8ffa1b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:31.744602 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd0c48-f9d6-482b-adce-306a8ffa1b77 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:31.745182 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd0c48-f9d6-482b-adce-306a8ffa1b77 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:31.745728 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd0c48-f9d6-482b-adce-306a8ffa1b77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:31.755804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd0c48-f9d6-482b-adce-306a8ffa1b77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:31.758016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd0c48-f9d6-482b-adce-306a8ffa1b77 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-b1bd0c48-f9d6-482b-adce-306a8ffa1b77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:31.785465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:31.786461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:31.788555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:31.788555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:31.788555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:31.788555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:31.788555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:31.788555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:31.788555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:31.788555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:31.788555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 389/797] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:31.789313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd0c48-f9d6-482b-adce-306a8ffa1b77 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:31.839676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb55a0-ed81-4572-837e-d51cd20b31f3 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:31.880158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f3486-b92b-4e01-ac4d-e1308eeb84c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:31.881105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f3486-b92b-4e01-ac4d-e1308eeb84c4 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:31.881380 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f3486-b92b-4e01-ac4d-e1308eeb84c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:31.881650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f3486-b92b-4e01-ac4d-e1308eeb84c4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:31.882279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f3486-b92b-4e01-ac4d-e1308eeb84c4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:31.883927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f3486-b92b-4e01-ac4d-e1308eeb84c4 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-752f3486-b92b-4e01-ac4d-e1308eeb84c4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:31.899780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:31.901014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:31.907105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 409/798] 10.222.0.86 () {56 vars in 1129 bytes} [Fri Aug 5 19:37:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:31.907989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21452523-0cb3-4127-88ce-34de25921b00 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:31.907989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21452523-0cb3-4127-88ce-34de25921b00 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:31.907989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21452523-0cb3-4127-88ce-34de25921b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:31.909446 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd0c48-f9d6-482b-adce-306a8ffa1b77 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:31.930053 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-21452523-0cb3-4127-88ce-34de25921b00 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:31.931273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-21452523-0cb3-4127-88ce-34de25921b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:31.950864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df728e0d-7e28-4da9-a45c-ff37ef9e79c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:31.951487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df728e0d-7e28-4da9-a45c-ff37ef9e79c0 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:31.951764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df728e0d-7e28-4da9-a45c-ff37ef9e79c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:31.952038 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df728e0d-7e28-4da9-a45c-ff37ef9e79c0 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:31.952757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df728e0d-7e28-4da9-a45c-ff37ef9e79c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:31.953060 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df728e0d-7e28-4da9-a45c-ff37ef9e79c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:31.954505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df728e0d-7e28-4da9-a45c-ff37ef9e79c0 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:31.963661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 390/799] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:37:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:31.964473 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df728e0d-7e28-4da9-a45c-ff37ef9e79c0 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:32.014657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 410/800] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:32.015744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21452523-0cb3-4127-88ce-34de25921b00 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:32.035197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:32.036581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:32.095833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df728e0d-7e28-4da9-a45c-ff37ef9e79c0 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:32.144414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ea03b-d017-4fa6-91c7-f53e009a17c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:32.145038 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ea03b-d017-4fa6-91c7-f53e009a17c5 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:32.145317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ea03b-d017-4fa6-91c7-f53e009a17c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:32.145589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ea03b-d017-4fa6-91c7-f53e009a17c5 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:32.146758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ea03b-d017-4fa6-91c7-f53e009a17c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:32.147071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ea03b-d017-4fa6-91c7-f53e009a17c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:32.148519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ea03b-d017-4fa6-91c7-f53e009a17c5 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:32.151199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21452523-0cb3-4127-88ce-34de25921b00 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:32.166944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 391/801] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:32.166944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ea03b-d017-4fa6-91c7-f53e009a17c5 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:32.200387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca783e-4b83-4765-83aa-40dacac7a8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:32.201139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca783e-4b83-4765-83aa-40dacac7a8df None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:32.201720 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca783e-4b83-4765-83aa-40dacac7a8df None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:32.201843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca783e-4b83-4765-83aa-40dacac7a8df None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:32.202527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca783e-4b83-4765-83aa-40dacac7a8df None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:32.202800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca783e-4b83-4765-83aa-40dacac7a8df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:32.204170 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca783e-4b83-4765-83aa-40dacac7a8df None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:32.210569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 411/802] 10.222.0.86 () {56 vars in 1124 bytes} [Fri Aug 5 19:37:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:32.217094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca783e-4b83-4765-83aa-40dacac7a8df None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:32.293719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ea03b-d017-4fa6-91c7-f53e009a17c5 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:32.309910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:32.315020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:32.337345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad3753-c199-47e4-bc99-ac22e78508f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:32.338320 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad3753-c199-47e4-bc99-ac22e78508f1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:32.338605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad3753-c199-47e4-bc99-ac22e78508f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:32.338909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad3753-c199-47e4-bc99-ac22e78508f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:32.347645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca783e-4b83-4765-83aa-40dacac7a8df None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:32.351279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad3753-c199-47e4-bc99-ac22e78508f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:32.352871 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad3753-c199-47e4-bc99-ac22e78508f1 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-e5ad3753-c199-47e4-bc99-ac22e78508f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:32.369170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:32.370348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:32.371299 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:32.371299 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:32.371299 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:32.371299 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:32.374989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 392/803] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:32.375791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad3753-c199-47e4-bc99-ac22e78508f1 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:32.387862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8214dcb-0079-4962-bcb5-fdba08597cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:32.388534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8214dcb-0079-4962-bcb5-fdba08597cca None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:32.388816 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8214dcb-0079-4962-bcb5-fdba08597cca None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:32.389087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8214dcb-0079-4962-bcb5-fdba08597cca None admin] PATH_INFO: `/v3/roles/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:32.401243 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8214dcb-0079-4962-bcb5-fdba08597cca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-c8214dcb-0079-4962-bcb5-fdba08597cca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:32.406566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:32.407975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 412/804] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:32.408817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8214dcb-0079-4962-bcb5-fdba08597cca None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:32.513474 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad3753-c199-47e4-bc99-ac22e78508f1 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:32.539555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8214dcb-0079-4962-bcb5-fdba08597cca None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:32.563401 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb8e9d-1cd9-4fc9-a90c-aab7170a2dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:32.564009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb8e9d-1cd9-4fc9-a90c-aab7170a2dad None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:32.564447 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb8e9d-1cd9-4fc9-a90c-aab7170a2dad None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:32.564716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb8e9d-1cd9-4fc9-a90c-aab7170a2dad None admin] PATH_INFO: `/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/e8adb0fd1fbc4268b70f3936036e550b/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:32.566774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb8e9d-1cd9-4fc9-a90c-aab7170a2dad None admin] RBAC: Authorizing `identity:create_grant(project_id=9c73d2cfaa1c47829ca6936186d2bb26, user_id=e8adb0fd1fbc4268b70f3936036e550b, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:32.568849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb8e9d-1cd9-4fc9-a90c-aab7170a2dad None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:32.580142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a521ac-e042-44d5-958b-8a652eb85a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:32.580675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a521ac-e042-44d5-958b-8a652eb85a57 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:32.580950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a521ac-e042-44d5-958b-8a652eb85a57 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:32.581220 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a521ac-e042-44d5-958b-8a652eb85a57 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:32.581902 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a521ac-e042-44d5-958b-8a652eb85a57 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:32.582203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a521ac-e042-44d5-958b-8a652eb85a57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:32.583615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a521ac-e042-44d5-958b-8a652eb85a57 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:32.585448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:32.585569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 393/805] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:37:32 2022] PUT /identity/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/e8adb0fd1fbc4268b70f3936036e550b/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:37:32.586364 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb8e9d-1cd9-4fc9-a90c-aab7170a2dad None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:32.601589 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 413/806] 10.222.0.86 () {56 vars in 1124 bytes} [Fri Aug 5 19:37:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:32.603562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a521ac-e042-44d5-958b-8a652eb85a57 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:32.627200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:32.736353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb8e9d-1cd9-4fc9-a90c-aab7170a2dad None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:32.767461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a521ac-e042-44d5-958b-8a652eb85a57 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:32.786668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd91542-7f63-471b-a5b6-12c6f34253eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:32.786668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd91542-7f63-471b-a5b6-12c6f34253eb None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:32.786668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd91542-7f63-471b-a5b6-12c6f34253eb None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:32.786668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd91542-7f63-471b-a5b6-12c6f34253eb None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:32.790649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd91542-7f63-471b-a5b6-12c6f34253eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:32.790649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd91542-7f63-471b-a5b6-12c6f34253eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:32.790649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd91542-7f63-471b-a5b6-12c6f34253eb None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:32.805273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 394/807] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:32.806514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd91542-7f63-471b-a5b6-12c6f34253eb None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:32.811389 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185ee81-65f0-43e3-88cb-f8a716c6897e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:32.812011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185ee81-65f0-43e3-88cb-f8a716c6897e None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:32.812290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185ee81-65f0-43e3-88cb-f8a716c6897e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:32.812563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185ee81-65f0-43e3-88cb-f8a716c6897e None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:32.813598 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185ee81-65f0-43e3-88cb-f8a716c6897e None admin] RBAC: Adding query filter params (name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:32.813844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185ee81-65f0-43e3-88cb-f8a716c6897e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:32.815256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185ee81-65f0-43e3-88cb-f8a716c6897e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:32.824904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 414/808] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:32.825774 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185ee81-65f0-43e3-88cb-f8a716c6897e None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:32.899210 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:32.927296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:32.948295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd91542-7f63-471b-a5b6-12c6f34253eb None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:32.954739 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185ee81-65f0-43e3-88cb-f8a716c6897e None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:32.985773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db2791-85fe-4591-9238-6ab6f21b3771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:32.986391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db2791-85fe-4591-9238-6ab6f21b3771 None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:32.986693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db2791-85fe-4591-9238-6ab6f21b3771 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:32.986962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db2791-85fe-4591-9238-6ab6f21b3771 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:32.987669 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db2791-85fe-4591-9238-6ab6f21b3771 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:32.989029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db2791-85fe-4591-9238-6ab6f21b3771 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:32.995338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae80b16-f884-4798-b892-c3ea6c9ad09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:32.996129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae80b16-f884-4798-b892-c3ea6c9ad09c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:32.996532 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae80b16-f884-4798-b892-c3ea6c9ad09c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:32.996986 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae80b16-f884-4798-b892-c3ea6c9ad09c None admin] PATH_INFO: `/v3/users/nova` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:33.006150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae80b16-f884-4798-b892-c3ea6c9ad09c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:33.007905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae80b16-f884-4798-b892-c3ea6c9ad09c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:33.017666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 395/809] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:37:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:37:33.027129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db2791-85fe-4591-9238-6ab6f21b3771 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-aae80b16-f884-4798-b892-c3ea6c9ad09c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:33.027221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:33.028472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:33.029585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:33.029585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:33.029585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:33.029585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:33.029585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:33.029585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:33.029585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:33.029585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:33.029585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:37:33.029585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:33.029585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 415/810] 10.222.0.86 () {56 vars in 1093 bytes} [Fri Aug 5 19:37:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:33.029585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1179f-f703-4167-bf12-c0ba4c86c097 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:33.029585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1179f-f703-4167-bf12-c0ba4c86c097 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:33.030379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1179f-f703-4167-bf12-c0ba4c86c097 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:33.030379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 416/811] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:33.030379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1179f-f703-4167-bf12-c0ba4c86c097 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:33.195141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db2791-85fe-4591-9238-6ab6f21b3771 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:33.195783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1179f-f703-4167-bf12-c0ba4c86c097 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:33.232879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:33.245482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cf3b3-ecd1-40e0-affa-6ceca64b2d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:33.246198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cf3b3-ecd1-40e0-affa-6ceca64b2d31 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:33.246485 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cf3b3-ecd1-40e0-affa-6ceca64b2d31 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:33.246798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cf3b3-ecd1-40e0-affa-6ceca64b2d31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:33.259619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:33.265282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79afc29-e7d0-4115-8106-afdea4d7cb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:33.266105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79afc29-e7d0-4115-8106-afdea4d7cb28 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:33.266407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79afc29-e7d0-4115-8106-afdea4d7cb28 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:33.267522 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79afc29-e7d0-4115-8106-afdea4d7cb28 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:33.267638 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cf3b3-ecd1-40e0-affa-6ceca64b2d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:33.267730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79afc29-e7d0-4115-8106-afdea4d7cb28 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:33.267815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79afc29-e7d0-4115-8106-afdea4d7cb28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:33.268582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cf3b3-ecd1-40e0-affa-6ceca64b2d31 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:33.269248 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79afc29-e7d0-4115-8106-afdea4d7cb28 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-314cf3b3-ecd1-40e0-affa-6ceca64b2d31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:33.292532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:33.293663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:33.295029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:33.295029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:33.295029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:33.295029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:33.295029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 396/812] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:33.295378 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168dc72-4523-458b-a3ee-bb341e76865a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:33.297505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168dc72-4523-458b-a3ee-bb341e76865a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:33.297505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168dc72-4523-458b-a3ee-bb341e76865a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:33.297685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 417/813] 10.222.0.86 () {56 vars in 1133 bytes} [Fri Aug 5 19:37:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:33.304399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79afc29-e7d0-4115-8106-afdea4d7cb28 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:33.321108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-2168dc72-4523-458b-a3ee-bb341e76865a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:33.322508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2168dc72-4523-458b-a3ee-bb341e76865a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:33.401119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 397/814] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:33.407588 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168dc72-4523-458b-a3ee-bb341e76865a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:33.414375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79afc29-e7d0-4115-8106-afdea4d7cb28 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:33.454420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e578c85-dbc1-446e-8a04-e5bc41f26641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:33.455098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578c85-dbc1-446e-8a04-e5bc41f26641 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:33.455380 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578c85-dbc1-446e-8a04-e5bc41f26641 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:33.455653 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578c85-dbc1-446e-8a04-e5bc41f26641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:33.509558 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:33.538108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e578c85-dbc1-446e-8a04-e5bc41f26641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:33.539941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e578c85-dbc1-446e-8a04-e5bc41f26641 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:33.553894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168dc72-4523-458b-a3ee-bb341e76865a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:33.577668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:33.594080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303dbfc-2a2b-4f69-8b20-7564520bc892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:33.596305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303dbfc-2a2b-4f69-8b20-7564520bc892 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:33.596587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303dbfc-2a2b-4f69-8b20-7564520bc892 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:33.596861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303dbfc-2a2b-4f69-8b20-7564520bc892 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:33.597693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303dbfc-2a2b-4f69-8b20-7564520bc892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:33.597998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303dbfc-2a2b-4f69-8b20-7564520bc892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:33.600094 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303dbfc-2a2b-4f69-8b20-7564520bc892 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:33.617456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 398/815] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:33.618384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303dbfc-2a2b-4f69-8b20-7564520bc892 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:33.665594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 418/816] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:37:33.667030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e578c85-dbc1-446e-8a04-e5bc41f26641 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:33.738964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303dbfc-2a2b-4f69-8b20-7564520bc892 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:33.752695 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:33.779719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23446cfc-0e88-4632-b677-17215dc8733f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:33.780667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23446cfc-0e88-4632-b677-17215dc8733f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:33.781125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23446cfc-0e88-4632-b677-17215dc8733f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:33.781573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23446cfc-0e88-4632-b677-17215dc8733f None admin] PATH_INFO: `/v3/services/network` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:33.782117 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e578c85-dbc1-446e-8a04-e5bc41f26641 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:33.782607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23446cfc-0e88-4632-b677-17215dc8733f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:33.784352 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23446cfc-0e88-4632-b677-17215dc8733f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-23446cfc-0e88-4632-b677-17215dc8733f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:33.800260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:33.801429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:37:33.802287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:33.812262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 399/817] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:33 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:33.815123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23446cfc-0e88-4632-b677-17215dc8733f None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:33.823086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ca7ff-3cbe-4ede-b30d-0306d4617738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:33.823802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ca7ff-3cbe-4ede-b30d-0306d4617738 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:33.824082 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ca7ff-3cbe-4ede-b30d-0306d4617738 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:33.824357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ca7ff-3cbe-4ede-b30d-0306d4617738 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:33.833116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ca7ff-3cbe-4ede-b30d-0306d4617738 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:33.835269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ca7ff-3cbe-4ede-b30d-0306d4617738 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-276ca7ff-3cbe-4ede-b30d-0306d4617738 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:33.848063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:33.849089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:33.851526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:33.851526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:33.851526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:33.851526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:33.851526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:33.851526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:33.851526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:33.851526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:33.851526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:37:33.851526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:33.851526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 419/818] 10.222.0.86 () {56 vars in 1117 bytes} [Fri Aug 5 19:37:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:33.851526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ca7ff-3cbe-4ede-b30d-0306d4617738 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:33.891151 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:33.986863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23446cfc-0e88-4632-b677-17215dc8733f None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:33.994257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ca7ff-3cbe-4ede-b30d-0306d4617738 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:34.036050 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb99e12-f799-4471-aa09-1477970654ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:34.036319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d7dd0-f2ee-4153-ba4b-7f19255cd81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:34.036794 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99e12-f799-4471-aa09-1477970654ea None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:34.036951 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d7dd0-f2ee-4153-ba4b-7f19255cd81c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:34.037098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99e12-f799-4471-aa09-1477970654ea None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:34.037383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d7dd0-f2ee-4153-ba4b-7f19255cd81c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:34.037515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99e12-f799-4471-aa09-1477970654ea None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:34.037747 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d7dd0-f2ee-4153-ba4b-7f19255cd81c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:34.038599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb99e12-f799-4471-aa09-1477970654ea None admin] RBAC: Adding query filter params (name=network) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:34.041880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb99e12-f799-4471-aa09-1477970654ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:34.041880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb99e12-f799-4471-aa09-1477970654ea None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:34.042916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:34.053682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d7dd0-f2ee-4153-ba4b-7f19255cd81c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:34.055544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 420/819] 10.222.0.86 () {56 vars in 1112 bytes} [Fri Aug 5 19:37:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:34.056555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb99e12-f799-4471-aa09-1477970654ea None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:34.056764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d7dd0-f2ee-4153-ba4b-7f19255cd81c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-3a6d7dd0-f2ee-4153-ba4b-7f19255cd81c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:34.076862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:34.078089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:34.079628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:34.079628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:34.079628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:34.079628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:34.081761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 400/820] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:34.082912 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d7dd0-f2ee-4153-ba4b-7f19255cd81c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:34.170558 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb99e12-f799-4471-aa09-1477970654ea None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:34.205941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:34.215302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d7dd0-f2ee-4153-ba4b-7f19255cd81c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:34.216247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79b730-03be-44d7-81fa-9018f0d2a295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:34.216870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79b730-03be-44d7-81fa-9018f0d2a295 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:34.217146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79b730-03be-44d7-81fa-9018f0d2a295 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:34.217414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79b730-03be-44d7-81fa-9018f0d2a295 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:34.218207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79b730-03be-44d7-81fa-9018f0d2a295 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:34.218506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79b730-03be-44d7-81fa-9018f0d2a295 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:34.220470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79b730-03be-44d7-81fa-9018f0d2a295 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:34.245454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 421/821] 10.222.0.86 () {56 vars in 1149 bytes} [Fri Aug 5 19:37:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:34.246833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79b730-03be-44d7-81fa-9018f0d2a295 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:34.265986 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cc074-083c-4d7c-9f5a-f67e70e0f0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:34.267078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cc074-083c-4d7c-9f5a-f67e70e0f0f5 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:34.267605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cc074-083c-4d7c-9f5a-f67e70e0f0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:34.268021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cc074-083c-4d7c-9f5a-f67e70e0f0f5 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:34.268868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cc074-083c-4d7c-9f5a-f67e70e0f0f5 None admin] RBAC: Adding query filter params (type=network) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:34.269336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cc074-083c-4d7c-9f5a-f67e70e0f0f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:34.271692 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cc074-083c-4d7c-9f5a-f67e70e0f0f5 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:34.278473 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 401/822] 10.222.0.86 () {56 vars in 1112 bytes} [Fri Aug 5 19:37:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:34.279682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cc074-083c-4d7c-9f5a-f67e70e0f0f5 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:34.323956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:34.374141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79b730-03be-44d7-81fa-9018f0d2a295 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:34.412515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583a6ec-e1ea-4fe5-82a6-b89bbfb34d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:34.413447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583a6ec-e1ea-4fe5-82a6-b89bbfb34d6e None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:34.413957 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583a6ec-e1ea-4fe5-82a6-b89bbfb34d6e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:34.414597 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583a6ec-e1ea-4fe5-82a6-b89bbfb34d6e None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:34.415526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583a6ec-e1ea-4fe5-82a6-b89bbfb34d6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:34.415694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583a6ec-e1ea-4fe5-82a6-b89bbfb34d6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:34.417043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583a6ec-e1ea-4fe5-82a6-b89bbfb34d6e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:34.429960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 422/823] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:34.431073 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583a6ec-e1ea-4fe5-82a6-b89bbfb34d6e None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:34.492074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cc074-083c-4d7c-9f5a-f67e70e0f0f5 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:34.552082 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583a6ec-e1ea-4fe5-82a6-b89bbfb34d6e None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:34.564544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0763a1e-5686-4b45-8881-00b5755e32cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:34.565929 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0763a1e-5686-4b45-8881-00b5755e32cc None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:34.566609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0763a1e-5686-4b45-8881-00b5755e32cc None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:34.567495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0763a1e-5686-4b45-8881-00b5755e32cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:34.572541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:34.592843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0763a1e-5686-4b45-8881-00b5755e32cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:34.594164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71019f-5a8f-44bf-9180-b73313475ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:34.594912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71019f-5a8f-44bf-9180-b73313475ddf None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:34.595188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71019f-5a8f-44bf-9180-b73313475ddf None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:34.595459 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71019f-5a8f-44bf-9180-b73313475ddf None admin] PATH_INFO: `/v3/endpoints` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:34.596341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71019f-5a8f-44bf-9180-b73313475ddf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:34.597026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0763a1e-5686-4b45-8881-00b5755e32cc None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:34.597706 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71019f-5a8f-44bf-9180-b73313475ddf None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-a0763a1e-5686-4b45-8881-00b5755e32cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:34.625694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:34.627439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:34.629687 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:34.629687 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:34.629687 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:34.629687 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:34.638706 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 423/825] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:37:34 2022] POST /identity/v3/endpoints => generated 337 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:37:34.638706 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71019f-5a8f-44bf-9180-b73313475ddf None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:34.639306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 402/825] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:34.642350 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0763a1e-5686-4b45-8881-00b5755e32cc None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:34.690162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:34.882192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71019f-5a8f-44bf-9180-b73313475ddf None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:34.887500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0763a1e-5686-4b45-8881-00b5755e32cc None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:34.925479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a8001-bdcf-45d9-9e1b-5a50760e03e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:34.926027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a8001-bdcf-45d9-9e1b-5a50760e03e3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:34.926295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a8001-bdcf-45d9-9e1b-5a50760e03e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:34.926565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a8001-bdcf-45d9-9e1b-5a50760e03e3 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:34.927981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a8001-bdcf-45d9-9e1b-5a50760e03e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:34.928396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a8001-bdcf-45d9-9e1b-5a50760e03e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:34.930588 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a8001-bdcf-45d9-9e1b-5a50760e03e3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:34.932139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177de2db-34c4-45d5-b1cd-80f2e6182c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:34.933007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177de2db-34c4-45d5-b1cd-80f2e6182c6b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:34.933521 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177de2db-34c4-45d5-b1cd-80f2e6182c6b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:34.942262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177de2db-34c4-45d5-b1cd-80f2e6182c6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:34.953964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 403/826] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:34.958125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a8001-bdcf-45d9-9e1b-5a50760e03e3 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:34.964491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177de2db-34c4-45d5-b1cd-80f2e6182c6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:34.966349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177de2db-34c4-45d5-b1cd-80f2e6182c6b None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-177de2db-34c4-45d5-b1cd-80f2e6182c6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:34.979771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:34.980819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:34.982019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:34.982019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:34.982019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:34.982019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:34.982019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:34.982019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:34.982019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:34.982019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:34.984420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 424/827] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:34.995792 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177de2db-34c4-45d5-b1cd-80f2e6182c6b None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:35.060893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:35.065662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:35.112240 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a8001-bdcf-45d9-9e1b-5a50760e03e3 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:35.158259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177de2db-34c4-45d5-b1cd-80f2e6182c6b None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:35.164061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba3918-aaa5-4d32-9980-94f3b0c31e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:35.164705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba3918-aaa5-4d32-9980-94f3b0c31e4b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:35.164983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba3918-aaa5-4d32-9980-94f3b0c31e4b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:35.165242 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba3918-aaa5-4d32-9980-94f3b0c31e4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:35.179608 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba3918-aaa5-4d32-9980-94f3b0c31e4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:35.181676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba3918-aaa5-4d32-9980-94f3b0c31e4b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-d7ba3918-aaa5-4d32-9980-94f3b0c31e4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:35.195961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:35.197082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:35.197992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:35.197992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:35.197992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:35.197992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:35.197992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:35.197992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:35.197992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:35.197992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:35.197992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 404/828] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:35.198769 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053bd7d-ec79-4504-9bb3-dd20bc52fd53 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:35.199041 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053bd7d-ec79-4504-9bb3-dd20bc52fd53 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:35.199310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053bd7d-ec79-4504-9bb3-dd20bc52fd53 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:35.200559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 405/829] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:35.202780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053bd7d-ec79-4504-9bb3-dd20bc52fd53 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:35.206040 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54e0b7-88be-4057-8363-3ff158adfb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:35.206947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54e0b7-88be-4057-8363-3ff158adfb70 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:35.207411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54e0b7-88be-4057-8363-3ff158adfb70 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:35.207861 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54e0b7-88be-4057-8363-3ff158adfb70 None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:35.209080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54e0b7-88be-4057-8363-3ff158adfb70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:35.209394 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54e0b7-88be-4057-8363-3ff158adfb70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:35.211740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54e0b7-88be-4057-8363-3ff158adfb70 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:35.226022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 425/830] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:35.228447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7df108-7493-434f-b7fb-976b3db49461 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:35.229343 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7df108-7493-434f-b7fb-976b3db49461 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:35.229827 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7df108-7493-434f-b7fb-976b3db49461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:35.253867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-7c7df108-7493-434f-b7fb-976b3db49461 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:35.255099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7c7df108-7493-434f-b7fb-976b3db49461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:35.327511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053bd7d-ec79-4504-9bb3-dd20bc52fd53 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:35.347748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 426/831] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:35.348846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7df108-7493-434f-b7fb-976b3db49461 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:35.369530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:35.372343 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:35.376841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209830df-e64c-4260-a627-c9240066f697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:35.377824 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209830df-e64c-4260-a627-c9240066f697 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:35.378286 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209830df-e64c-4260-a627-c9240066f697 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:35.378738 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209830df-e64c-4260-a627-c9240066f697 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:35.380014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209830df-e64c-4260-a627-c9240066f697 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:35.380563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209830df-e64c-4260-a627-c9240066f697 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:35.382416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209830df-e64c-4260-a627-c9240066f697 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:35.394934 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 406/832] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:35.396269 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750734b-f8b7-4aeb-b088-5a6483cb7224 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:35.396894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750734b-f8b7-4aeb-b088-5a6483cb7224 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:35.396894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750734b-f8b7-4aeb-b088-5a6483cb7224 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:35.398298 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 407/833] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:35.398851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750734b-f8b7-4aeb-b088-5a6483cb7224 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:35.466889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7df108-7493-434f-b7fb-976b3db49461 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:35.506642 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d368947-6635-4c96-aa39-a77d4389afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:35.507326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d368947-6635-4c96-aa39-a77d4389afcd None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:35.507605 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d368947-6635-4c96-aa39-a77d4389afcd None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:35.507875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d368947-6635-4c96-aa39-a77d4389afcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:35.508675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d368947-6635-4c96-aa39-a77d4389afcd None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=ab8f9d9cf4ca4e38ae6eb67e949b40fd) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:35.508993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d368947-6635-4c96-aa39-a77d4389afcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:35.510431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d368947-6635-4c96-aa39-a77d4389afcd None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:35.514516 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750734b-f8b7-4aeb-b088-5a6483cb7224 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:35.516920 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 427/834] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:35 2022] GET /identity/v3/role_assignments?user.id=ab8f9d9cf4ca4e38ae6eb67e949b40fd&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=93eb204498fc49d297bcd1841e00818d => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:35.517718 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d368947-6635-4c96-aa39-a77d4389afcd None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:35.563697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ed144-bc65-4997-af53-d9af843881b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:35.564907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ed144-bc65-4997-af53-d9af843881b5 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:35.565388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ed144-bc65-4997-af53-d9af843881b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:35.565842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ed144-bc65-4997-af53-d9af843881b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:35.579458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ed144-bc65-4997-af53-d9af843881b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-e73ed144-bc65-4997-af53-d9af843881b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:35.586058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:35.587890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 408/835] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:37:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:35.588735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d824e-03d7-4865-a425-2de89027a523 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:35.589494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d824e-03d7-4865-a425-2de89027a523 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:35.589494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d824e-03d7-4865-a425-2de89027a523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:35.609785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-119d824e-03d7-4865-a425-2de89027a523 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:35.611547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-119d824e-03d7-4865-a425-2de89027a523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:35.636491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:35.659571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d368947-6635-4c96-aa39-a77d4389afcd None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:35.675905 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:35.708928 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 409/836] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:35.709248 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d383e8-be99-4d70-8b55-7c8a6f724440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:35.709969 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d824e-03d7-4865-a425-2de89027a523 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:35.710357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d383e8-be99-4d70-8b55-7c8a6f724440 None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:35.710909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d383e8-be99-4d70-8b55-7c8a6f724440 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:35.711361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d383e8-be99-4d70-8b55-7c8a6f724440 None admin] PATH_INFO: `/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/7fc4868f25364311a504224fcd2c5a73/roles/02001b18565340ce984343c5d36daaf9` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:35.732601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d383e8-be99-4d70-8b55-7c8a6f724440 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c73d2cfaa1c47829ca6936186d2bb26, user_id=7fc4868f25364311a504224fcd2c5a73, role_id=02001b18565340ce984343c5d36daaf9)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:35.732601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d383e8-be99-4d70-8b55-7c8a6f724440 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:35.750806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 428/837] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:37:35 2022] PUT /identity/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/7fc4868f25364311a504224fcd2c5a73/roles/02001b18565340ce984343c5d36daaf9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:37:35.751655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d383e8-be99-4d70-8b55-7c8a6f724440 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:35.877293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d824e-03d7-4865-a425-2de89027a523 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:35.922323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062152ef-a469-46bc-8c7d-f6349238f641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:35.922976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062152ef-a469-46bc-8c7d-f6349238f641 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:35.923254 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062152ef-a469-46bc-8c7d-f6349238f641 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:35.923521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062152ef-a469-46bc-8c7d-f6349238f641 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:35.924234 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062152ef-a469-46bc-8c7d-f6349238f641 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:35.924529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062152ef-a469-46bc-8c7d-f6349238f641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:35.924854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d383e8-be99-4d70-8b55-7c8a6f724440 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:35.926012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062152ef-a469-46bc-8c7d-f6349238f641 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:35.942305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 410/838] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:37:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:35.949708 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062152ef-a469-46bc-8c7d-f6349238f641 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:35.970503 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:35.977767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e1281-2307-492c-bf80-3748a4f86dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:35.978440 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e1281-2307-492c-bf80-3748a4f86dc3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:35.978759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e1281-2307-492c-bf80-3748a4f86dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:35.979031 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e1281-2307-492c-bf80-3748a4f86dc3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:35.979676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e1281-2307-492c-bf80-3748a4f86dc3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:35.981056 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e1281-2307-492c-bf80-3748a4f86dc3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-412e1281-2307-492c-bf80-3748a4f86dc3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:35.994359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:35.996137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:35.997777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 429/839] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:37:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:36.002913 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e1281-2307-492c-bf80-3748a4f86dc3 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:36.031993 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:36.111945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062152ef-a469-46bc-8c7d-f6349238f641 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:36.132206 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e1281-2307-492c-bf80-3748a4f86dc3 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:36.158530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e386c2-c233-413c-8a76-d2c5c2bbfa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:36.159334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e386c2-c233-413c-8a76-d2c5c2bbfa9e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:36.159616 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e386c2-c233-413c-8a76-d2c5c2bbfa9e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:36.159886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e386c2-c233-413c-8a76-d2c5c2bbfa9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:36.173197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e386c2-c233-413c-8a76-d2c5c2bbfa9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:36.174805 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e386c2-c233-413c-8a76-d2c5c2bbfa9e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:36.179015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14ddec-53b9-4e31-bec5-8a4772924055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:36.179653 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14ddec-53b9-4e31-bec5-8a4772924055 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:36.179930 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14ddec-53b9-4e31-bec5-8a4772924055 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:36.180204 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14ddec-53b9-4e31-bec5-8a4772924055 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:36.180890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14ddec-53b9-4e31-bec5-8a4772924055 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:36.181193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14ddec-53b9-4e31-bec5-8a4772924055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:36.182740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14ddec-53b9-4e31-bec5-8a4772924055 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:36.188545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 430/840] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:37:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-f4e386c2-c233-413c-8a76-d2c5c2bbfa9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:36.195092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:36.196495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:36.198048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:36.198048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:36.198048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:36.198048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:36.198048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 411/841] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:36.198601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14ddec-53b9-4e31-bec5-8a4772924055 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:36.202262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e386c2-c233-413c-8a76-d2c5c2bbfa9e None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:36.285455 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:36.319595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:36.326371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e386c2-c233-413c-8a76-d2c5c2bbfa9e None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:36.339350 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14ddec-53b9-4e31-bec5-8a4772924055 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:36.374329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53162ac1-770f-4108-81fa-114cce26f647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:36.374329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53162ac1-770f-4108-81fa-114cce26f647 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:36.374329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53162ac1-770f-4108-81fa-114cce26f647 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:36.374329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53162ac1-770f-4108-81fa-114cce26f647 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:36.374845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53162ac1-770f-4108-81fa-114cce26f647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:36.374845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53162ac1-770f-4108-81fa-114cce26f647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:36.374845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53162ac1-770f-4108-81fa-114cce26f647 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:36.388566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0365-3166-4052-beaa-0ef44ee1c83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:36.389349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0365-3166-4052-beaa-0ef44ee1c83c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:36.389733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0365-3166-4052-beaa-0ef44ee1c83c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:36.390109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0365-3166-4052-beaa-0ef44ee1c83c None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:36.390552 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 412/842] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:36.394329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed0365-3166-4052-beaa-0ef44ee1c83c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:36.394866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed0365-3166-4052-beaa-0ef44ee1c83c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:36.398652 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53162ac1-770f-4108-81fa-114cce26f647 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:36.398983 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed0365-3166-4052-beaa-0ef44ee1c83c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:36.402935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 431/843] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:37:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:36.531595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53162ac1-770f-4108-81fa-114cce26f647 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:36.574416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f13871-219e-443e-9009-c9660b7e31ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:36.575055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f13871-219e-443e-9009-c9660b7e31ea None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:36.575339 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f13871-219e-443e-9009-c9660b7e31ea None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:36.575612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f13871-219e-443e-9009-c9660b7e31ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:36.584683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f13871-219e-443e-9009-c9660b7e31ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:36.586273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f13871-219e-443e-9009-c9660b7e31ea None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-42f13871-219e-443e-9009-c9660b7e31ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:36.600233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:36.601312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:36.602930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:36.602930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:36.602930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:36.602930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:36.602930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:36.602930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:36.602930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:36.602930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:36.602930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:37:36.602930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:36.602930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 413/844] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:37:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:36.607456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0365-3166-4052-beaa-0ef44ee1c83c None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:36.728135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0365-3166-4052-beaa-0ef44ee1c83c None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:36.767003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:36.769600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf88aa-13e5-4d6d-a9b1-c81274b32ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:36.770223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf88aa-13e5-4d6d-a9b1-c81274b32ee6 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:36.770803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf88aa-13e5-4d6d-a9b1-c81274b32ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:36.771168 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf88aa-13e5-4d6d-a9b1-c81274b32ee6 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:36.772697 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf88aa-13e5-4d6d-a9b1-c81274b32ee6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:36.773002 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf88aa-13e5-4d6d-a9b1-c81274b32ee6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:36.774587 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf88aa-13e5-4d6d-a9b1-c81274b32ee6 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:36.797275 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 432/845] 10.222.0.86 () {56 vars in 1135 bytes} [Fri Aug 5 19:37:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:36.804415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f13871-219e-443e-9009-c9660b7e31ea None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:36.825913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:36.934472 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f13871-219e-443e-9009-c9660b7e31ea None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:36.976486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33c3ad-1b34-41f0-b4ab-80a329a30089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:36.977119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33c3ad-1b34-41f0-b4ab-80a329a30089 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:36.977651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33c3ad-1b34-41f0-b4ab-80a329a30089 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:36.977931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33c3ad-1b34-41f0-b4ab-80a329a30089 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:37.054120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33c3ad-1b34-41f0-b4ab-80a329a30089 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:37.055681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33c3ad-1b34-41f0-b4ab-80a329a30089 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:37.074999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:37.168529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 414/846] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:37:37.173818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf88aa-13e5-4d6d-a9b1-c81274b32ee6 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:37.288195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf88aa-13e5-4d6d-a9b1-c81274b32ee6 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:37.325572 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d499314-c3a9-4b13-81d4-63717d9025a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:37.326218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d499314-c3a9-4b13-81d4-63717d9025a1 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:37.326577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d499314-c3a9-4b13-81d4-63717d9025a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:37.326874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d499314-c3a9-4b13-81d4-63717d9025a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:37.338329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d499314-c3a9-4b13-81d4-63717d9025a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:37.339833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d499314-c3a9-4b13-81d4-63717d9025a1 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-9d499314-c3a9-4b13-81d4-63717d9025a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:37.358354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:37.359354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:37.360678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:37.360678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:37.360678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:37.360678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:37.360678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 433/847] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:37.365945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33c3ad-1b34-41f0-b4ab-80a329a30089 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:37.482073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33c3ad-1b34-41f0-b4ab-80a329a30089 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:37.502661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:37.520866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdde6e-7c17-4d34-a56f-7dfd5083f58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:37.522205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdde6e-7c17-4d34-a56f-7dfd5083f58c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:37.522585 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdde6e-7c17-4d34-a56f-7dfd5083f58c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:37.522889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdde6e-7c17-4d34-a56f-7dfd5083f58c None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:37.523593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdde6e-7c17-4d34-a56f-7dfd5083f58c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:37.523894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdde6e-7c17-4d34-a56f-7dfd5083f58c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:37.525274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdde6e-7c17-4d34-a56f-7dfd5083f58c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:37.538161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 415/848] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:37.543971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d499314-c3a9-4b13-81d4-63717d9025a1 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:37.607535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:37.658305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d499314-c3a9-4b13-81d4-63717d9025a1 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:37.702880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ece09a-7761-4fd7-8ab4-64bbca36baf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:37.703530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ece09a-7761-4fd7-8ab4-64bbca36baf9 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:37.703806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ece09a-7761-4fd7-8ab4-64bbca36baf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:37.704075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ece09a-7761-4fd7-8ab4-64bbca36baf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:37.712198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ece09a-7761-4fd7-8ab4-64bbca36baf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:37.713664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ece09a-7761-4fd7-8ab4-64bbca36baf9 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-65ece09a-7761-4fd7-8ab4-64bbca36baf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:37.726826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:37.727822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:37.729131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:37.729131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:37.729131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:37.729131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:37.729131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:37.729131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:37.729131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:37.729131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:37.729131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 434/849] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:37.734181 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdde6e-7c17-4d34-a56f-7dfd5083f58c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:37.749465 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7037181-cc63-4967-87b6-6e4eb009c36d None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:37.749766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7037181-cc63-4967-87b6-6e4eb009c36d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:37.750070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7037181-cc63-4967-87b6-6e4eb009c36d None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:37.751606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 435/850] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:37.759335 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff9427-3e90-46c7-a57c-727600389fc8 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:37.759773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff9427-3e90-46c7-a57c-727600389fc8 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:37.760374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff9427-3e90-46c7-a57c-727600389fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:37.787382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-99ff9427-3e90-46c7-a57c-727600389fc8 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:37.788840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-99ff9427-3e90-46c7-a57c-727600389fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:37.873281 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdde6e-7c17-4d34-a56f-7dfd5083f58c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:37.886649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 436/851] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:37.903891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff9427-3e90-46c7-a57c-727600389fc8 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:37.920644 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334ecde-0add-4d76-9f0a-1dd2a7e3ce44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:37.921652 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334ecde-0add-4d76-9f0a-1dd2a7e3ce44 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:37.922223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334ecde-0add-4d76-9f0a-1dd2a7e3ce44 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:37.922812 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334ecde-0add-4d76-9f0a-1dd2a7e3ce44 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:37.923837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334ecde-0add-4d76-9f0a-1dd2a7e3ce44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:37.924359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334ecde-0add-4d76-9f0a-1dd2a7e3ce44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:37.926113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334ecde-0add-4d76-9f0a-1dd2a7e3ce44 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:37.935189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:37.948464 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 416/852] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:37.955251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334ecde-0add-4d76-9f0a-1dd2a7e3ce44 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:38.003091 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:38.028921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff9427-3e90-46c7-a57c-727600389fc8 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:38.071404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e98c1-2a90-4fdc-8514-a10067ea1891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:38.072445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e98c1-2a90-4fdc-8514-a10067ea1891 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:38.072951 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e98c1-2a90-4fdc-8514-a10067ea1891 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:38.073431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e98c1-2a90-4fdc-8514-a10067ea1891 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:38.074278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e98c1-2a90-4fdc-8514-a10067ea1891 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:38.075905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e98c1-2a90-4fdc-8514-a10067ea1891 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-0e3e98c1-2a90-4fdc-8514-a10067ea1891 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:38.086802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:38.087866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:37:38.088733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:38.096328 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334ecde-0add-4d76-9f0a-1dd2a7e3ce44 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:38.096686 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 437/853] 10.222.0.86 () {56 vars in 1132 bytes} [Fri Aug 5 19:37:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:38.097850 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e98c1-2a90-4fdc-8514-a10067ea1891 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:38.141465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0ed9a-2cac-4605-8ea2-90ca2d945415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:38.142084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0ed9a-2cac-4605-8ea2-90ca2d945415 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:38.142359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0ed9a-2cac-4605-8ea2-90ca2d945415 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:38.142651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0ed9a-2cac-4605-8ea2-90ca2d945415 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:38.143442 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0ed9a-2cac-4605-8ea2-90ca2d945415 None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=e8adb0fd1fbc4268b70f3936036e550b) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:38.143749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0ed9a-2cac-4605-8ea2-90ca2d945415 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:38.145177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0ed9a-2cac-4605-8ea2-90ca2d945415 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:38.152524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 417/854] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:37 2022] GET /identity/v3/role_assignments?user.id=e8adb0fd1fbc4268b70f3936036e550b&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=93eb204498fc49d297bcd1841e00818d => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:38.153781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbbb5d-1cf5-49fa-8229-97c84237551d None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:38.154064 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbbb5d-1cf5-49fa-8229-97c84237551d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:38.154401 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbbb5d-1cf5-49fa-8229-97c84237551d None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:38.155988 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 418/855] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:38.162614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4ee05-5dcc-41ef-9871-a8736c31d903 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:38.162864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4ee05-5dcc-41ef-9871-a8736c31d903 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:38.163095 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4ee05-5dcc-41ef-9871-a8736c31d903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:38.184438 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-76e4ee05-5dcc-41ef-9871-a8736c31d903 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:38.185895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-76e4ee05-5dcc-41ef-9871-a8736c31d903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:38.237903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:38.252941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e98c1-2a90-4fdc-8514-a10067ea1891 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:38.300350 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 419/856] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:38.308011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86649efd-8908-46ad-9973-d185993070a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:38.308661 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86649efd-8908-46ad-9973-d185993070a2 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:38.308955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86649efd-8908-46ad-9973-d185993070a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:38.309211 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86649efd-8908-46ad-9973-d185993070a2 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:38.309921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86649efd-8908-46ad-9973-d185993070a2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:38.310300 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86649efd-8908-46ad-9973-d185993070a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:38.311878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86649efd-8908-46ad-9973-d185993070a2 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:38.317906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 438/857] 10.222.0.86 () {56 vars in 1126 bytes} [Fri Aug 5 19:37:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:38.324107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4ee05-5dcc-41ef-9871-a8736c31d903 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:38.325791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86649efd-8908-46ad-9973-d185993070a2 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:38.368439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:38.456765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86649efd-8908-46ad-9973-d185993070a2 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:38.461334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4ee05-5dcc-41ef-9871-a8736c31d903 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:38.504110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560c873-1d98-4ffc-ab1d-b19867e2e274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:38.504790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560c873-1d98-4ffc-ab1d-b19867e2e274 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:38.505121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560c873-1d98-4ffc-ab1d-b19867e2e274 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:38.505436 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560c873-1d98-4ffc-ab1d-b19867e2e274 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:38.510552 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a4dc3-bb0d-409d-ba07-d60e1ce14b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:38.511769 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a4dc3-bb0d-409d-ba07-d60e1ce14b7f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:38.512355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a4dc3-bb0d-409d-ba07-d60e1ce14b7f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:38.512881 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a4dc3-bb0d-409d-ba07-d60e1ce14b7f None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:38.513803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a4dc3-bb0d-409d-ba07-d60e1ce14b7f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:38.514466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a4dc3-bb0d-409d-ba07-d60e1ce14b7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:38.516409 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a4dc3-bb0d-409d-ba07-d60e1ce14b7f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:38.518441 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560c873-1d98-4ffc-ab1d-b19867e2e274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:38.523990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 420/858] 10.222.0.86 () {56 vars in 1126 bytes} [Fri Aug 5 19:37:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-1560c873-1d98-4ffc-ab1d-b19867e2e274 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:38.527639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:38.528615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:38.529448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:38.529448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:38.529448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:38.529448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:38.529448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:38.529448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:38.529448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:38.529448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:38.529448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:38.529448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:38.529448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:37:38.529448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:38.532432 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 439/859] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:37:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:38.540395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a4dc3-bb0d-409d-ba07-d60e1ce14b7f None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:38.674191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:38.681909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644162d-b9d8-4f72-8155-cb5a6b990a36 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:38.682484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644162d-b9d8-4f72-8155-cb5a6b990a36 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:38.682484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644162d-b9d8-4f72-8155-cb5a6b990a36 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:38.683312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a4dc3-bb0d-409d-ba07-d60e1ce14b7f None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:38.688813 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 440/860] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:38.690651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3fc37-dbca-427f-8962-5c5bdf1a9e7a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:38.691002 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3fc37-dbca-427f-8962-5c5bdf1a9e7a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:38.691352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3fc37-dbca-427f-8962-5c5bdf1a9e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:38.716166 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-b2d3fc37-dbca-427f-8962-5c5bdf1a9e7a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:38.717795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b2d3fc37-dbca-427f-8962-5c5bdf1a9e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:38.731055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5fe69-275b-4995-ba11-0988a5fea2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:38.731737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5fe69-275b-4995-ba11-0988a5fea2a6 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:38.732018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5fe69-275b-4995-ba11-0988a5fea2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:38.732247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:38.732350 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5fe69-275b-4995-ba11-0988a5fea2a6 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:38.733003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5fe69-275b-4995-ba11-0988a5fea2a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:38.733306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5fe69-275b-4995-ba11-0988a5fea2a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:38.734997 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5fe69-275b-4995-ba11-0988a5fea2a6 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:38.745662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 421/861] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:37:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:38.751711 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5fe69-275b-4995-ba11-0988a5fea2a6 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:38.820438 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 441/862] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:38.843729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3fc37-dbca-427f-8962-5c5bdf1a9e7a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:38.876668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5fe69-275b-4995-ba11-0988a5fea2a6 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:38.916792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b20d59-ac95-4272-8e71-713a8a2ce4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:38.917791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b20d59-ac95-4272-8e71-713a8a2ce4bb None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:38.918270 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b20d59-ac95-4272-8e71-713a8a2ce4bb None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:38.918776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b20d59-ac95-4272-8e71-713a8a2ce4bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:38.931765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b20d59-ac95-4272-8e71-713a8a2ce4bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:38.933575 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b20d59-ac95-4272-8e71-713a8a2ce4bb None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-a1b20d59-ac95-4272-8e71-713a8a2ce4bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:38.952120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:38.953324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:38.954399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:38.954399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:38.954399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:38.954399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:38.961390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 422/863] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:38.961617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3fc37-dbca-427f-8962-5c5bdf1a9e7a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:38.962924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b20d59-ac95-4272-8e71-713a8a2ce4bb None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:38.975987 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:39.004108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd6a8f-dc4a-45d9-8bb6-0716cb65042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:39.005150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd6a8f-dc4a-45d9-8bb6-0716cb65042a None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:39.006305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd6a8f-dc4a-45d9-8bb6-0716cb65042a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:39.007010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd6a8f-dc4a-45d9-8bb6-0716cb65042a None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:39.007966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd6a8f-dc4a-45d9-8bb6-0716cb65042a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:39.009684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd6a8f-dc4a-45d9-8bb6-0716cb65042a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:39.030059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 442/864] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:37:38 2022] POST /identity/v3/services => generated 251 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:37:39.104953 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b20d59-ac95-4272-8e71-713a8a2ce4bb None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:39.160998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183e837-7018-4e17-8394-1fc767ef2b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:39.163326 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183e837-7018-4e17-8394-1fc767ef2b79 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:39.163823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183e837-7018-4e17-8394-1fc767ef2b79 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:39.164310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183e837-7018-4e17-8394-1fc767ef2b79 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:39.165255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183e837-7018-4e17-8394-1fc767ef2b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:39.165794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183e837-7018-4e17-8394-1fc767ef2b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:39.167818 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183e837-7018-4e17-8394-1fc767ef2b79 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:39.188800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 423/865] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:39.195510 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd6a8f-dc4a-45d9-8bb6-0716cb65042a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:39.200463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:39.323184 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd6a8f-dc4a-45d9-8bb6-0716cb65042a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:39.365862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191bd5c-96ec-48c2-bd48-f201005cacf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:39.366617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191bd5c-96ec-48c2-bd48-f201005cacf7 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:39.366917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191bd5c-96ec-48c2-bd48-f201005cacf7 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:39.367189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7191bd5c-96ec-48c2-bd48-f201005cacf7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:39.378991 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191bd5c-96ec-48c2-bd48-f201005cacf7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:39.381106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7191bd5c-96ec-48c2-bd48-f201005cacf7 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-7191bd5c-96ec-48c2-bd48-f201005cacf7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:39.394570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:39.395613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:39.396617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:39.396617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:39.396617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:39.396617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:39.396617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:39.396617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:39.396617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:39.396617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:39.396617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:37:39.396617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:39.396617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 443/866] 10.222.0.86 () {56 vars in 1117 bytes} [Fri Aug 5 19:37:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:39.404578 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183e837-7018-4e17-8394-1fc767ef2b79 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:39.482888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:39.529244 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183e837-7018-4e17-8394-1fc767ef2b79 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:39.567169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed79c08-6f2f-4263-80ed-30b878903a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:39.567793 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed79c08-6f2f-4263-80ed-30b878903a16 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:39.568670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed79c08-6f2f-4263-80ed-30b878903a16 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:39.568950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed79c08-6f2f-4263-80ed-30b878903a16 None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:39.569795 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed79c08-6f2f-4263-80ed-30b878903a16 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:39.570102 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed79c08-6f2f-4263-80ed-30b878903a16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:39.571592 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed79c08-6f2f-4263-80ed-30b878903a16 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:39.592295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 424/867] 10.222.0.86 () {56 vars in 1149 bytes} [Fri Aug 5 19:37:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:39.597663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191bd5c-96ec-48c2-bd48-f201005cacf7 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:39.671458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:39.722667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7191bd5c-96ec-48c2-bd48-f201005cacf7 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:39.777984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376429ce-2904-4374-886e-f632bcfc7eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:39.779143 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376429ce-2904-4374-886e-f632bcfc7eb6 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:39.779657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376429ce-2904-4374-886e-f632bcfc7eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:39.780134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376429ce-2904-4374-886e-f632bcfc7eb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:39.860294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376429ce-2904-4374-886e-f632bcfc7eb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:39.861826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376429ce-2904-4374-886e-f632bcfc7eb6 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:39.930536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:39.975322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 444/868] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:39 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:37:39.984045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed79c08-6f2f-4263-80ed-30b878903a16 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:40.118100 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed79c08-6f2f-4263-80ed-30b878903a16 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:40.153501 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:40.169338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4267af5-ca84-456a-9444-398b4182b5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:40.169982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4267af5-ca84-456a-9444-398b4182b5e4 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:40.170253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4267af5-ca84-456a-9444-398b4182b5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:40.170518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4267af5-ca84-456a-9444-398b4182b5e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:40.182933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4267af5-ca84-456a-9444-398b4182b5e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:40.184395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4267af5-ca84-456a-9444-398b4182b5e4 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-e4267af5-ca84-456a-9444-398b4182b5e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:40.201853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:40.202851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:40.204829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:40.204829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:40.204829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:40.204829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:40.204829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 425/869] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:40.209174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376429ce-2904-4374-886e-f632bcfc7eb6 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:40.325928 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376429ce-2904-4374-886e-f632bcfc7eb6 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:40.365770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009f301-b1bb-4961-b7a7-f463abf50bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:40.366446 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009f301-b1bb-4961-b7a7-f463abf50bc2 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:40.366800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009f301-b1bb-4961-b7a7-f463abf50bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:40.367069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009f301-b1bb-4961-b7a7-f463abf50bc2 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:40.367770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009f301-b1bb-4961-b7a7-f463abf50bc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:40.368064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009f301-b1bb-4961-b7a7-f463abf50bc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:40.369500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009f301-b1bb-4961-b7a7-f463abf50bc2 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:40.384980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 445/870] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:40.391009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4267af5-ca84-456a-9444-398b4182b5e4 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:40.422518 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c505c-d910-4c4e-8e76-93c7a2bf3240 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:40.422883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c505c-d910-4c4e-8e76-93c7a2bf3240 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:40.423100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c505c-d910-4c4e-8e76-93c7a2bf3240 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:40.424776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 446/871] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:37:40.430490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2e972-aa73-4cab-a5bf-9b567826225c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:40.430838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2e972-aa73-4cab-a5bf-9b567826225c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:40.431072 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2e972-aa73-4cab-a5bf-9b567826225c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:40.453623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-eed2e972-aa73-4cab-a5bf-9b567826225c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:40.454920 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-eed2e972-aa73-4cab-a5bf-9b567826225c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:40.480094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:40.526129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4267af5-ca84-456a-9444-398b4182b5e4 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:40.547107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 447/872] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:37:40.566521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4b2bb-33f5-4759-a6a8-6790025d1df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:40.567194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4b2bb-33f5-4759-a6a8-6790025d1df0 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:40.567468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4b2bb-33f5-4759-a6a8-6790025d1df0 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:40.567739 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4b2bb-33f5-4759-a6a8-6790025d1df0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:40.574579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2e972-aa73-4cab-a5bf-9b567826225c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:40.575775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4b2bb-33f5-4759-a6a8-6790025d1df0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:40.577434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4b2bb-33f5-4759-a6a8-6790025d1df0 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-04e4b2bb-33f5-4759-a6a8-6790025d1df0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:40.593381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:40.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:40.595534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:40.595534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:40.595534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:40.595534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:40.595534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:40.595534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:40.595534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:40.595534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:40.595534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 426/873] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:40.601685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4b2bb-33f5-4759-a6a8-6790025d1df0 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:40.641218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:40.704956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2e972-aa73-4cab-a5bf-9b567826225c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:40.718583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4b2bb-33f5-4759-a6a8-6790025d1df0 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:40.744546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fab46-b3fa-4d02-a92e-8a1588821e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:40.745466 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fab46-b3fa-4d02-a92e-8a1588821e88 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:40.745875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fab46-b3fa-4d02-a92e-8a1588821e88 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:40.746219 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55fab46-b3fa-4d02-a92e-8a1588821e88 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:40.747032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fab46-b3fa-4d02-a92e-8a1588821e88 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:40.748717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55fab46-b3fa-4d02-a92e-8a1588821e88 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:40.758037 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809731b-59ba-46e0-9627-d9bfd89cc0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:40.758808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809731b-59ba-46e0-9627-d9bfd89cc0b8 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:40.759149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809731b-59ba-46e0-9627-d9bfd89cc0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:40.759530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809731b-59ba-46e0-9627-d9bfd89cc0b8 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-a55fab46-b3fa-4d02-a92e-8a1588821e88 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:40.759684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:40.761662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:40.763801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:40.765593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809731b-59ba-46e0-9627-d9bfd89cc0b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:40.765593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809731b-59ba-46e0-9627-d9bfd89cc0b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:40.765593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809731b-59ba-46e0-9627-d9bfd89cc0b8 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:40.770677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 448/874] 10.222.0.86 () {56 vars in 1126 bytes} [Fri Aug 5 19:37:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:40.770677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fab46-b3fa-4d02-a92e-8a1588821e88 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:40.784508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 427/875] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:40.785719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809731b-59ba-46e0-9627-d9bfd89cc0b8 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:40.797945 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:40.893445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55fab46-b3fa-4d02-a92e-8a1588821e88 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:40.895750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:40.904932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809731b-59ba-46e0-9627-d9bfd89cc0b8 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:40.943866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58808004-7dba-4594-8963-2c1cd8bc1a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:40.944529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808004-7dba-4594-8963-2c1cd8bc1a2d None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:40.944809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808004-7dba-4594-8963-2c1cd8bc1a2d None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:40.945082 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58808004-7dba-4594-8963-2c1cd8bc1a2d None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:40.945767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808004-7dba-4594-8963-2c1cd8bc1a2d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:40.946074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808004-7dba-4594-8963-2c1cd8bc1a2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:40.947506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58808004-7dba-4594-8963-2c1cd8bc1a2d None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:40.948050 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35976c-afb1-47fd-aeb4-4879a5628236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:40.948708 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35976c-afb1-47fd-aeb4-4879a5628236 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:40.948939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35976c-afb1-47fd-aeb4-4879a5628236 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:40.949200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35976c-afb1-47fd-aeb4-4879a5628236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:40.950100 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35976c-afb1-47fd-aeb4-4879a5628236 None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=7fc4868f25364311a504224fcd2c5a73) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:40.950286 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35976c-afb1-47fd-aeb4-4879a5628236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:40.951745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35976c-afb1-47fd-aeb4-4879a5628236 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:40.953652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 449/876] 10.222.0.86 () {56 vars in 1122 bytes} [Fri Aug 5 19:37:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:40.954927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904fe67-a2fb-4de7-834b-deac30a78ded None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:40.955208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904fe67-a2fb-4de7-834b-deac30a78ded None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:40.955481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904fe67-a2fb-4de7-834b-deac30a78ded None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:40.961336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 450/877] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:40.961336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904fe67-a2fb-4de7-834b-deac30a78ded None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:40.964371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 428/878] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:40 2022] GET /identity/v3/role_assignments?user.id=7fc4868f25364311a504224fcd2c5a73&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:40.968758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c9d15-1cca-4819-a4e6-c31ef93db3fa None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:40.968758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c9d15-1cca-4819-a4e6-c31ef93db3fa None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:40.968758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c9d15-1cca-4819-a4e6-c31ef93db3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:40.990972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-365c9d15-1cca-4819-a4e6-c31ef93db3fa None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:40.993468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-365c9d15-1cca-4819-a4e6-c31ef93db3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:41.082060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:41.109396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 429/879] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:41.111047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904fe67-a2fb-4de7-834b-deac30a78ded None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:41.127521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c9d15-1cca-4819-a4e6-c31ef93db3fa None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:41.169851 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836cb17-88be-4bcf-a93e-6c6653f53201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:41.170463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836cb17-88be-4bcf-a93e-6c6653f53201 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:41.170769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836cb17-88be-4bcf-a93e-6c6653f53201 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:41.171036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836cb17-88be-4bcf-a93e-6c6653f53201 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:41.172130 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836cb17-88be-4bcf-a93e-6c6653f53201 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:41.172433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836cb17-88be-4bcf-a93e-6c6653f53201 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:41.174098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836cb17-88be-4bcf-a93e-6c6653f53201 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:41.180571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 451/880] 10.222.0.86 () {56 vars in 1122 bytes} [Fri Aug 5 19:37:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:41.250127 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:41.268515 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c9d15-1cca-4819-a4e6-c31ef93db3fa None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:41.329159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256dba1-a8d0-4418-9f90-520a173ec66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:41.330177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256dba1-a8d0-4418-9f90-520a173ec66c None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:41.330667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256dba1-a8d0-4418-9f90-520a173ec66c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:41.331255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256dba1-a8d0-4418-9f90-520a173ec66c None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:41.332580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256dba1-a8d0-4418-9f90-520a173ec66c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:41.335447 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256dba1-a8d0-4418-9f90-520a173ec66c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:41.366232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645afd0-2b94-468c-82cd-c608655ed4c6 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:41.366730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645afd0-2b94-468c-82cd-c608655ed4c6 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:41.367191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645afd0-2b94-468c-82cd-c608655ed4c6 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:41.369624 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 452/881] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:37:41.373609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 430/882] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:37:41 2022] POST /identity/v3/services => generated 269 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:37:41.376102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804a9db-b73f-4451-b638-609612a9a637 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:41.376459 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804a9db-b73f-4451-b638-609612a9a637 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:41.376825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804a9db-b73f-4451-b638-609612a9a637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:41.402330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3804a9db-b73f-4451-b638-609612a9a637 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:41.403634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3804a9db-b73f-4451-b638-609612a9a637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:41.507303 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 453/883] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:37:41.530469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256dba1-a8d0-4418-9f90-520a173ec66c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:41.662826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256dba1-a8d0-4418-9f90-520a173ec66c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:41.703373 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e716f3f-5540-41d8-8979-2f3b677b6824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:41.704415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e716f3f-5540-41d8-8979-2f3b677b6824 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:41.704928 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e716f3f-5540-41d8-8979-2f3b677b6824 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:41.705416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e716f3f-5540-41d8-8979-2f3b677b6824 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:41.706255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e716f3f-5540-41d8-8979-2f3b677b6824 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:41.707931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e716f3f-5540-41d8-8979-2f3b677b6824 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-2e716f3f-5540-41d8-8979-2f3b677b6824 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:41.718892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:41.720023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:37:41.720868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:41.727768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 431/884] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:37:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:41.731272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804a9db-b73f-4451-b638-609612a9a637 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:41.814353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:41.858423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804a9db-b73f-4451-b638-609612a9a637 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:41.902286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d5559-fd89-4432-8b96-5d82e5cea086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:41.903319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d5559-fd89-4432-8b96-5d82e5cea086 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:41.903793 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d5559-fd89-4432-8b96-5d82e5cea086 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:41.904338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d5559-fd89-4432-8b96-5d82e5cea086 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:41.905534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d5559-fd89-4432-8b96-5d82e5cea086 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:41.906002 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d5559-fd89-4432-8b96-5d82e5cea086 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:41.907443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d5559-fd89-4432-8b96-5d82e5cea086 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:41.915021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 454/885] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:37:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:41.920914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e716f3f-5540-41d8-8979-2f3b677b6824 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:41.940536 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:42.049857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e716f3f-5540-41d8-8979-2f3b677b6824 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:42.089404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2ac4b-6d54-44f2-a2ee-313378b7ce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:42.090413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2ac4b-6d54-44f2-a2ee-313378b7ce9c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:42.090928 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2ac4b-6d54-44f2-a2ee-313378b7ce9c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:42.091403 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2ac4b-6d54-44f2-a2ee-313378b7ce9c None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:42.092583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2ac4b-6d54-44f2-a2ee-313378b7ce9c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:42.093180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2ac4b-6d54-44f2-a2ee-313378b7ce9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:42.094804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d2ac4b-6d54-44f2-a2ee-313378b7ce9c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:42.106757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 432/886] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:37:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:42.108891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d5559-fd89-4432-8b96-5d82e5cea086 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:42.242799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d5559-fd89-4432-8b96-5d82e5cea086 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:42.281109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8ab2e-dd83-4abe-b2b4-2b3968785ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:42.281734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8ab2e-dd83-4abe-b2b4-2b3968785ed3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:42.282009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8ab2e-dd83-4abe-b2b4-2b3968785ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:42.282278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8ab2e-dd83-4abe-b2b4-2b3968785ed3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:42.283056 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8ab2e-dd83-4abe-b2b4-2b3968785ed3 None admin] RBAC: Adding query filter params (interface=public, service_id=4e0d4d76e99e4a3db4e8d1c5f5047a35, region_id=RegionOne) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:42.283358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8ab2e-dd83-4abe-b2b4-2b3968785ed3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:42.284726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8ab2e-dd83-4abe-b2b4-2b3968785ed3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:42.286322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:42.297867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 455/887] 10.222.0.86 () {56 vars in 1251 bytes} [Fri Aug 5 19:37:42 2022] GET /identity/v3/endpoints?service_id=4e0d4d76e99e4a3db4e8d1c5f5047a35&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:42.299074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2ac4b-6d54-44f2-a2ee-313378b7ce9c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:42.395445 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:42.424719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2ac4b-6d54-44f2-a2ee-313378b7ce9c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:42.465856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a2c89-e90d-45a5-bd28-1bb5ce0736d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:42.466935 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a2c89-e90d-45a5-bd28-1bb5ce0736d4 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:42.467429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a2c89-e90d-45a5-bd28-1bb5ce0736d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:42.467936 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a2c89-e90d-45a5-bd28-1bb5ce0736d4 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:42.468851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a2c89-e90d-45a5-bd28-1bb5ce0736d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:42.470528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a2c89-e90d-45a5-bd28-1bb5ce0736d4 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:42.484690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 433/888] 10.222.0.86 () {56 vars in 1087 bytes} [Fri Aug 5 19:37:42 2022] GET /identity/v3/services => generated 1268 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:37:43.336013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c3b6b-bc73-4301-82df-0259c0f98342 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:43.336617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c3b6b-bc73-4301-82df-0259c0f98342 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:43.336617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c3b6b-bc73-4301-82df-0259c0f98342 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:43.338217 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 456/889] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:43.343898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e08324-65cd-4eef-b793-c3d1f9731664 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:43.344378 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e08324-65cd-4eef-b793-c3d1f9731664 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:43.344455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e08324-65cd-4eef-b793-c3d1f9731664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:43.363998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-83e08324-65cd-4eef-b793-c3d1f9731664 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:43.365238 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-83e08324-65cd-4eef-b793-c3d1f9731664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:43.456484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 434/890] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:43.480781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c3b6b-bc73-4301-82df-0259c0f98342 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:43.496617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9ff9d-8c95-48bd-b469-ce475691dabd None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:43.497301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9ff9d-8c95-48bd-b469-ce475691dabd None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:43.497783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9ff9d-8c95-48bd-b469-ce475691dabd None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:43.500175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 435/891] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:43.508085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3a63e-1dbf-428c-9e37-a15ef43aa0dc None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:43.508718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3a63e-1dbf-428c-9e37-a15ef43aa0dc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:43.509212 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3a63e-1dbf-428c-9e37-a15ef43aa0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:43.530909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-ace3a63e-1dbf-428c-9e37-a15ef43aa0dc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:43.532513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ace3a63e-1dbf-428c-9e37-a15ef43aa0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:43.615386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:43.615845 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c3b6b-bc73-4301-82df-0259c0f98342 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:43.616197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 436/892] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:43.628794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b9318-1142-47d6-a7d2-6b0e6df534fb None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:43.629432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b9318-1142-47d6-a7d2-6b0e6df534fb None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:43.629961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b9318-1142-47d6-a7d2-6b0e6df534fb None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:43.632517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 437/893] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:43.634044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b9318-1142-47d6-a7d2-6b0e6df534fb None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:43.663194 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cef441-18dc-4f93-ae50-0451c33c54dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:43.665740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cef441-18dc-4f93-ae50-0451c33c54dc None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:43.665740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cef441-18dc-4f93-ae50-0451c33c54dc None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:43.665740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cef441-18dc-4f93-ae50-0451c33c54dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:43.685805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cef441-18dc-4f93-ae50-0451c33c54dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-e3cef441-18dc-4f93-ae50-0451c33c54dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:43.691059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:43.692095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:43.693036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:43.693036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:43.693036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:43.693036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:43.693036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:43.693036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:43.693036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:43.693036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:43.693036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:43.693036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:43.693036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:37:43.693036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:43.693036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 457/894] 10.222.0.86 () {56 vars in 1120 bytes} [Fri Aug 5 19:37:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:43.694818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb635a-c42f-4310-887a-0251f1000230 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:43.694818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb635a-c42f-4310-887a-0251f1000230 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:43.695017 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb635a-c42f-4310-887a-0251f1000230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:43.726136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-90fb635a-c42f-4310-887a-0251f1000230 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:43.726766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-90fb635a-c42f-4310-887a-0251f1000230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:43.784005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b9318-1142-47d6-a7d2-6b0e6df534fb None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:43.804992 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:43.822564 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 458/895] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:43.823941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb635a-c42f-4310-887a-0251f1000230 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:43.839555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052d264-f3e0-4fbc-9b64-83ecd407ab8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:43.841065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052d264-f3e0-4fbc-9b64-83ecd407ab8c None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:43.841729 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052d264-f3e0-4fbc-9b64-83ecd407ab8c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:43.842169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052d264-f3e0-4fbc-9b64-83ecd407ab8c None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:43.843313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052d264-f3e0-4fbc-9b64-83ecd407ab8c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:43.845551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052d264-f3e0-4fbc-9b64-83ecd407ab8c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:43.863750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 438/896] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:37:43 2022] POST /identity/v3/services => generated 241 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:37:43.864414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052d264-f3e0-4fbc-9b64-83ecd407ab8c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:43.946550 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:43.967989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb635a-c42f-4310-887a-0251f1000230 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:44.009045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052d264-f3e0-4fbc-9b64-83ecd407ab8c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:44.011254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c9ab7-9517-49af-b87b-0f9bb214fb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:44.011752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c9ab7-9517-49af-b87b-0f9bb214fb96 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:44.012033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c9ab7-9517-49af-b87b-0f9bb214fb96 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:44.012305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c9ab7-9517-49af-b87b-0f9bb214fb96 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:44.013772 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c9ab7-9517-49af-b87b-0f9bb214fb96 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:44.013772 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c9ab7-9517-49af-b87b-0f9bb214fb96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:44.014782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c9ab7-9517-49af-b87b-0f9bb214fb96 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:44.030515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 459/897] 10.222.0.86 () {56 vars in 1115 bytes} [Fri Aug 5 19:37:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:44.033323 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c9ab7-9517-49af-b87b-0f9bb214fb96 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:44.049869 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8bbef-ff77-43ed-9b20-6ed56c44c093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:44.050530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8bbef-ff77-43ed-9b20-6ed56c44c093 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:44.050863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8bbef-ff77-43ed-9b20-6ed56c44c093 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:44.051135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8bbef-ff77-43ed-9b20-6ed56c44c093 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:44.051794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8bbef-ff77-43ed-9b20-6ed56c44c093 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:44.053818 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8bbef-ff77-43ed-9b20-6ed56c44c093 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-33b8bbef-ff77-43ed-9b20-6ed56c44c093 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:44.064835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:44.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:37:44.066727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:44.070704 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 439/898] 10.222.0.86 () {56 vars in 1132 bytes} [Fri Aug 5 19:37:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:44.073191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8bbef-ff77-43ed-9b20-6ed56c44c093 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:44.102820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:44.158095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c9ab7-9517-49af-b87b-0f9bb214fb96 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:44.198647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7326a4-c10b-4218-84ba-abf92bcb5847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:44.199642 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7326a4-c10b-4218-84ba-abf92bcb5847 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:44.200024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7326a4-c10b-4218-84ba-abf92bcb5847 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:44.200828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7326a4-c10b-4218-84ba-abf92bcb5847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:44.218188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7326a4-c10b-4218-84ba-abf92bcb5847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:44.220218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7326a4-c10b-4218-84ba-abf92bcb5847 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:44.229456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8bbef-ff77-43ed-9b20-6ed56c44c093 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-4a7326a4-c10b-4218-84ba-abf92bcb5847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:44.247911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:44.248876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:44.249767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:44.249767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:44.249767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:44.249767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:44.256382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 460/899] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:44.259194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:44.260150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7326a4-c10b-4218-84ba-abf92bcb5847 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:44.276082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e19b1-247b-4b0d-8851-4d6919e12469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:44.276811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e19b1-247b-4b0d-8851-4d6919e12469 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:44.277206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e19b1-247b-4b0d-8851-4d6919e12469 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:44.277484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e19b1-247b-4b0d-8851-4d6919e12469 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:44.278170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e19b1-247b-4b0d-8851-4d6919e12469 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:44.278473 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e19b1-247b-4b0d-8851-4d6919e12469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:44.279960 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e19b1-247b-4b0d-8851-4d6919e12469 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:44.290838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 440/900] 10.222.0.86 () {56 vars in 1126 bytes} [Fri Aug 5 19:37:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:44.293003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e19b1-247b-4b0d-8851-4d6919e12469 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:44.409467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e19b1-247b-4b0d-8851-4d6919e12469 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:44.450350 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7326a4-c10b-4218-84ba-abf92bcb5847 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:44.453557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed3ecc-a446-4930-820c-a674149d0698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:44.455369 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed3ecc-a446-4930-820c-a674149d0698 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:44.455890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed3ecc-a446-4930-820c-a674149d0698 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:44.456365 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed3ecc-a446-4930-820c-a674149d0698 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:44.457253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed3ecc-a446-4930-820c-a674149d0698 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:44.457789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed3ecc-a446-4930-820c-a674149d0698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:44.460392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed3ecc-a446-4930-820c-a674149d0698 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:44.467136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 441/901] 10.222.0.86 () {56 vars in 1126 bytes} [Fri Aug 5 19:37:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:44.468248 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:44.473769 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed3ecc-a446-4930-820c-a674149d0698 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:44.514182 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4e651-b6c3-415a-b3c4-131d19d33a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:44.515597 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4e651-b6c3-415a-b3c4-131d19d33a3f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:44.516886 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4e651-b6c3-415a-b3c4-131d19d33a3f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:44.517482 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4e651-b6c3-415a-b3c4-131d19d33a3f None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:44.519167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4e651-b6c3-415a-b3c4-131d19d33a3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:44.519884 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4e651-b6c3-415a-b3c4-131d19d33a3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:44.522456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4e651-b6c3-415a-b3c4-131d19d33a3f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:44.543515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 461/902] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:44.551894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4e651-b6c3-415a-b3c4-131d19d33a3f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:44.573461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:44.629052 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed3ecc-a446-4930-820c-a674149d0698 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:44.666518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369beb5-0323-4ab7-ab9d-a5bd22b493b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:44.667327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369beb5-0323-4ab7-ab9d-a5bd22b493b1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:44.667603 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369beb5-0323-4ab7-ab9d-a5bd22b493b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:44.667873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369beb5-0323-4ab7-ab9d-a5bd22b493b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:44.668632 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369beb5-0323-4ab7-ab9d-a5bd22b493b1 None admin] RBAC: Adding query filter params (interface=public, service_id=76239a53206f41b78e5a18637f134834, region_id=RegionOne) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:44.668939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369beb5-0323-4ab7-ab9d-a5bd22b493b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:44.670305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369beb5-0323-4ab7-ab9d-a5bd22b493b1 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:44.680989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 442/903] 10.222.0.86 () {56 vars in 1251 bytes} [Fri Aug 5 19:37:44 2022] GET /identity/v3/endpoints?service_id=76239a53206f41b78e5a18637f134834&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:44.682525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369beb5-0323-4ab7-ab9d-a5bd22b493b1 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:44.752166 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4e651-b6c3-415a-b3c4-131d19d33a3f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:44.792467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369beb5-0323-4ab7-ab9d-a5bd22b493b1 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:44.807319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:44.818138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2300c3-cc88-48df-ba87-9a0e6bd331fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:44.819354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2300c3-cc88-48df-ba87-9a0e6bd331fd None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:44.819803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2300c3-cc88-48df-ba87-9a0e6bd331fd None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:44.820219 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2300c3-cc88-48df-ba87-9a0e6bd331fd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:44.831900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2300c3-cc88-48df-ba87-9a0e6bd331fd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:44.834125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2300c3-cc88-48df-ba87-9a0e6bd331fd None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:44.834692 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e22efe-690f-4a3e-880b-080dd5a3f4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:44.835586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e22efe-690f-4a3e-880b-080dd5a3f4d0 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:44.835860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e22efe-690f-4a3e-880b-080dd5a3f4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:44.836130 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e22efe-690f-4a3e-880b-080dd5a3f4d0 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:44.836783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e22efe-690f-4a3e-880b-080dd5a3f4d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:44.838503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e22efe-690f-4a3e-880b-080dd5a3f4d0 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-4c2300c3-cc88-48df-ba87-9a0e6bd331fd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:44.850542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:44.851632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:44.852651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:44.852651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:44.852651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:44.852651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:44.852651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:44.852651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:44.852651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:44.852651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:44.852651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:37:44.852651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:44.853267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 443/904] 10.222.0.86 () {56 vars in 1087 bytes} [Fri Aug 5 19:37:44 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:37:44.855283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 462/905] 10.222.0.86 () {56 vars in 1117 bytes} [Fri Aug 5 19:37:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:44.866654 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e22efe-690f-4a3e-880b-080dd5a3f4d0 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:44.988602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c91c10-57db-4e67-aaa1-1df409422fa7 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:44.988905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c91c10-57db-4e67-aaa1-1df409422fa7 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:44.989146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c91c10-57db-4e67-aaa1-1df409422fa7 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:44.991406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 463/906] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:37:44.997078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6303163-243d-43bb-a591-fd462d22b878 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:44.997373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6303163-243d-43bb-a591-fd462d22b878 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:44.997621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6303163-243d-43bb-a591-fd462d22b878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:44.998842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e22efe-690f-4a3e-880b-080dd5a3f4d0 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:45.022554 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c6303163-243d-43bb-a591-fd462d22b878 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:45.024240 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c6303163-243d-43bb-a591-fd462d22b878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:45.049223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:45.050232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90781ca4-7b63-4eac-8efc-86d85302ab9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:45.051163 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90781ca4-7b63-4eac-8efc-86d85302ab9a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:45.051493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90781ca4-7b63-4eac-8efc-86d85302ab9a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:45.051803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90781ca4-7b63-4eac-8efc-86d85302ab9a None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:45.052949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90781ca4-7b63-4eac-8efc-86d85302ab9a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:45.053258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90781ca4-7b63-4eac-8efc-86d85302ab9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:45.054730 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90781ca4-7b63-4eac-8efc-86d85302ab9a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:45.081449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 444/907] 10.222.0.86 () {56 vars in 1149 bytes} [Fri Aug 5 19:37:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:45.087494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90781ca4-7b63-4eac-8efc-86d85302ab9a None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:45.117379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 464/908] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:45.134690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:45.139770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6303163-243d-43bb-a591-fd462d22b878 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:45.221537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90781ca4-7b63-4eac-8efc-86d85302ab9a None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:45.268261 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ae2db-f9ad-4bbb-b56c-02720a56254f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:45.268890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ae2db-f9ad-4bbb-b56c-02720a56254f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:45.269171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ae2db-f9ad-4bbb-b56c-02720a56254f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:45.269600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ae2db-f9ad-4bbb-b56c-02720a56254f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:45.280624 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6303163-243d-43bb-a591-fd462d22b878 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:45.319911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e70353-94be-4062-a0a4-1d70b7360603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:45.320563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e70353-94be-4062-a0a4-1d70b7360603 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:45.320840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e70353-94be-4062-a0a4-1d70b7360603 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:45.321110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e70353-94be-4062-a0a4-1d70b7360603 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:45.321776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e70353-94be-4062-a0a4-1d70b7360603 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:45.323259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e70353-94be-4062-a0a4-1d70b7360603 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-62e70353-94be-4062-a0a4-1d70b7360603 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:45.333172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:45.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:45.336397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 465/909] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:37:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:45.344013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e70353-94be-4062-a0a4-1d70b7360603 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:45.356992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ae2db-f9ad-4bbb-b56c-02720a56254f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:45.358614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ae2db-f9ad-4bbb-b56c-02720a56254f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:45.371555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:45.407438 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:45.488189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e70353-94be-4062-a0a4-1d70b7360603 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:45.518656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 445/910] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:45 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:37:45.527568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ae2db-f9ad-4bbb-b56c-02720a56254f None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:45.543522 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9aeee-ffde-4e76-85bc-c035b20485a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:45.544515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9aeee-ffde-4e76-85bc-c035b20485a7 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:45.545022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9aeee-ffde-4e76-85bc-c035b20485a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:45.545499 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9aeee-ffde-4e76-85bc-c035b20485a7 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:45.546395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9aeee-ffde-4e76-85bc-c035b20485a7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:45.546937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9aeee-ffde-4e76-85bc-c035b20485a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:45.549569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9aeee-ffde-4e76-85bc-c035b20485a7 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:45.560891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 466/911] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:37:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:45.562172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9aeee-ffde-4e76-85bc-c035b20485a7 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:45.645872 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ae2db-f9ad-4bbb-b56c-02720a56254f None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:45.658785 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:45.683631 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9aeee-ffde-4e76-85bc-c035b20485a7 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:45.688451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a194d-bcf0-46d5-b0a5-6959d83dc870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:45.689186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a194d-bcf0-46d5-b0a5-6959d83dc870 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:45.689537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a194d-bcf0-46d5-b0a5-6959d83dc870 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:45.689881 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a194d-bcf0-46d5-b0a5-6959d83dc870 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:45.693559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:45.705527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a194d-bcf0-46d5-b0a5-6959d83dc870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:45.707426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a194d-bcf0-46d5-b0a5-6959d83dc870 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:45.726819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327919c-a4c1-442d-8939-cf0f8120097f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-748a194d-bcf0-46d5-b0a5-6959d83dc870 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:45.727033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:45.728003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:45.728965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:45.728965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:45.728965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:45.728965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:45.730185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327919c-a4c1-442d-8939-cf0f8120097f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:45.730656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327919c-a4c1-442d-8939-cf0f8120097f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:45.731093 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327919c-a4c1-442d-8939-cf0f8120097f None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:45.731991 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327919c-a4c1-442d-8939-cf0f8120097f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:45.732534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327919c-a4c1-442d-8939-cf0f8120097f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:45.733001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 446/912] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:45.733970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a194d-bcf0-46d5-b0a5-6959d83dc870 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:45.734758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327919c-a4c1-442d-8939-cf0f8120097f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:45.746388 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 467/913] 10.222.0.86 () {56 vars in 1114 bytes} [Fri Aug 5 19:37:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:45.747686 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327919c-a4c1-442d-8939-cf0f8120097f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:45.854563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a194d-bcf0-46d5-b0a5-6959d83dc870 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:45.876589 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327919c-a4c1-442d-8939-cf0f8120097f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:45.894482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a480e-aece-43df-9086-1e25b9ea7d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:45.895716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a480e-aece-43df-9086-1e25b9ea7d93 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:45.896853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a480e-aece-43df-9086-1e25b9ea7d93 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:45.897314 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a480e-aece-43df-9086-1e25b9ea7d93 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:45.898698 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a480e-aece-43df-9086-1e25b9ea7d93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:45.899105 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a480e-aece-43df-9086-1e25b9ea7d93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:45.900632 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a480e-aece-43df-9086-1e25b9ea7d93 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:45.914318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 447/914] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:45.917724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366711f2-d4f9-481b-9dc5-07420ec7a6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:45.918603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366711f2-d4f9-481b-9dc5-07420ec7a6c0 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:45.919203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366711f2-d4f9-481b-9dc5-07420ec7a6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:45.919692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366711f2-d4f9-481b-9dc5-07420ec7a6c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:45.919904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a480e-aece-43df-9086-1e25b9ea7d93 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:45.920814 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366711f2-d4f9-481b-9dc5-07420ec7a6c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:45.922416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366711f2-d4f9-481b-9dc5-07420ec7a6c0 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:45.941609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:45.959443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 468/915] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:37:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:37:45.991044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:46.064660 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a480e-aece-43df-9086-1e25b9ea7d93 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:46.113412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a8ef7-454c-4343-b209-07bd568d30dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:46.113996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a8ef7-454c-4343-b209-07bd568d30dc None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:46.114271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a8ef7-454c-4343-b209-07bd568d30dc None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:46.114536 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a8ef7-454c-4343-b209-07bd568d30dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:46.124344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a8ef7-454c-4343-b209-07bd568d30dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:46.125823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a8ef7-454c-4343-b209-07bd568d30dc None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-a95a8ef7-454c-4343-b209-07bd568d30dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:46.139388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:46.140716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:46.141856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:46.141856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:46.141856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:46.141856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:46.141856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:46.141856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:46.141856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:46.141856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:46.141856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 448/916] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:46.147144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366711f2-d4f9-481b-9dc5-07420ec7a6c0 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:46.192705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30422b9-c105-4b91-837a-09855d65a2a6 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:46.193521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30422b9-c105-4b91-837a-09855d65a2a6 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:46.193776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30422b9-c105-4b91-837a-09855d65a2a6 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:46.195508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 449/917] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:46.201611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118e173-f91f-434a-a3f2-cefaf2714d20 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:46.201902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118e173-f91f-434a-a3f2-cefaf2714d20 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:46.202155 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d118e173-f91f-434a-a3f2-cefaf2714d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:46.221981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-d118e173-f91f-434a-a3f2-cefaf2714d20 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:46.223216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d118e173-f91f-434a-a3f2-cefaf2714d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:46.287237 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366711f2-d4f9-481b-9dc5-07420ec7a6c0 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:46.322803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 450/918] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:46.340487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa2922-2ffd-4a6d-a0c6-f398670dd578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:46.341899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa2922-2ffd-4a6d-a0c6-f398670dd578 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:46.342580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa2922-2ffd-4a6d-a0c6-f398670dd578 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:46.343310 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa2922-2ffd-4a6d-a0c6-f398670dd578 None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:46.344309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa2922-2ffd-4a6d-a0c6-f398670dd578 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:46.344815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa2922-2ffd-4a6d-a0c6-f398670dd578 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:46.346138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118e173-f91f-434a-a3f2-cefaf2714d20 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:46.346719 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa2922-2ffd-4a6d-a0c6-f398670dd578 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:46.362938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 469/919] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:46.369688 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa2922-2ffd-4a6d-a0c6-f398670dd578 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:46.392691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:46.448889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:46.501933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa2922-2ffd-4a6d-a0c6-f398670dd578 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:46.502845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d118e173-f91f-434a-a3f2-cefaf2714d20 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:46.545734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c503c-cb94-4b01-93d8-c759376077e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:46.547345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c503c-cb94-4b01-93d8-c759376077e4 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:46.547832 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c503c-cb94-4b01-93d8-c759376077e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:46.548425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c503c-cb94-4b01-93d8-c759376077e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:46.549303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c503c-cb94-4b01-93d8-c759376077e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:46.550907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c503c-cb94-4b01-93d8-c759376077e4 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:46.551726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b544b-51d9-46af-b774-8907ae5018ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:46.552663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b544b-51d9-46af-b774-8907ae5018ac None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:46.552944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b544b-51d9-46af-b774-8907ae5018ac None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:46.553219 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b544b-51d9-46af-b774-8907ae5018ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:46.554329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b544b-51d9-46af-b774-8907ae5018ac None admin] RBAC: Adding query filter params (role.id=6b65af6de416463283285acc9f0f40f9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=7fc4868f25364311a504224fcd2c5a73) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:46.554670 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b544b-51d9-46af-b774-8907ae5018ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:46.557881 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b544b-51d9-46af-b774-8907ae5018ac None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-a35c503c-cb94-4b01-93d8-c759376077e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:46.563582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:46.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:46.565531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:46.573162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 470/920] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:46 2022] GET /identity/v3/role_assignments?user.id=7fc4868f25364311a504224fcd2c5a73&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=6b65af6de416463283285acc9f0f40f9 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:46.573815 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 451/921] 10.222.0.86 () {56 vars in 1126 bytes} [Fri Aug 5 19:37:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:46.576123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b544b-51d9-46af-b774-8907ae5018ac None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:46.698770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b544b-51d9-46af-b774-8907ae5018ac None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:46.728563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:46.742307 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b9e84-7985-405a-9c34-e89b66e82faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:46.742969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b9e84-7985-405a-9c34-e89b66e82faa None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:46.743247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b9e84-7985-405a-9c34-e89b66e82faa None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:46.743525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b9e84-7985-405a-9c34-e89b66e82faa None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:46.744292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b9e84-7985-405a-9c34-e89b66e82faa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:46.744610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b9e84-7985-405a-9c34-e89b66e82faa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:46.745999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b9e84-7985-405a-9c34-e89b66e82faa None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:46.751764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 471/922] 10.222.0.86 () {56 vars in 1122 bytes} [Fri Aug 5 19:37:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:46.759897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c503c-cb94-4b01-93d8-c759376077e4 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:46.877473 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:46.884772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c503c-cb94-4b01-93d8-c759376077e4 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:46.925976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6d7a3-8f6b-43b7-8273-9176f9678da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:46.926597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6d7a3-8f6b-43b7-8273-9176f9678da3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:46.926900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6d7a3-8f6b-43b7-8273-9176f9678da3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:46.928284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6d7a3-8f6b-43b7-8273-9176f9678da3 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:46.928980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d6d7a3-8f6b-43b7-8273-9176f9678da3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:46.929281 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d6d7a3-8f6b-43b7-8273-9176f9678da3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:46.930691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d6d7a3-8f6b-43b7-8273-9176f9678da3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:46.936443 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 452/923] 10.222.0.86 () {56 vars in 1122 bytes} [Fri Aug 5 19:37:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:46.943319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b9e84-7985-405a-9c34-e89b66e82faa None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:47.055139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b9e84-7985-405a-9c34-e89b66e82faa None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:47.092277 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dce173-7947-4516-a0eb-ffca94e4ecc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:47.092890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dce173-7947-4516-a0eb-ffca94e4ecc8 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:47.093165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dce173-7947-4516-a0eb-ffca94e4ecc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:47.093432 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dce173-7947-4516-a0eb-ffca94e4ecc8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:47.094191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dce173-7947-4516-a0eb-ffca94e4ecc8 None admin] RBAC: Adding query filter params (interface=public, service_id=a53f38790f6f4890a5505982c9812ce9, region_id=RegionOne) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:47.094495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dce173-7947-4516-a0eb-ffca94e4ecc8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:47.095895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dce173-7947-4516-a0eb-ffca94e4ecc8 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:47.102481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 472/924] 10.222.0.86 () {56 vars in 1251 bytes} [Fri Aug 5 19:37:46 2022] GET /identity/v3/endpoints?service_id=a53f38790f6f4890a5505982c9812ce9&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:47.108213 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6d7a3-8f6b-43b7-8273-9176f9678da3 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:47.142917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6817c6-dfe9-4165-89a4-d343ebefb60c None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:47.143282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6817c6-dfe9-4165-89a4-d343ebefb60c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:47.143473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6817c6-dfe9-4165-89a4-d343ebefb60c None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:47.145120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 473/925] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:47.150781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd74d5c-0389-4c81-9ce7-1470f409e172 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:47.151090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd74d5c-0389-4c81-9ce7-1470f409e172 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:47.151342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd74d5c-0389-4c81-9ce7-1470f409e172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:47.176341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-dfd74d5c-0389-4c81-9ce7-1470f409e172 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:47.178168 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-dfd74d5c-0389-4c81-9ce7-1470f409e172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:47.248345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:47.252369 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6d7a3-8f6b-43b7-8273-9176f9678da3 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:47.282921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 474/926] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:37:47.301849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd74d5c-0389-4c81-9ce7-1470f409e172 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:47.306177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b5132-d41d-416b-a88e-9a08aa0e7fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:47.306837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5132-d41d-416b-a88e-9a08aa0e7fab None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:47.307255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5132-d41d-416b-a88e-9a08aa0e7fab None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:47.307539 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5132-d41d-416b-a88e-9a08aa0e7fab None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:47.308450 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b5132-d41d-416b-a88e-9a08aa0e7fab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:47.310062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b5132-d41d-416b-a88e-9a08aa0e7fab None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:47.317717 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 453/927] 10.222.0.86 () {56 vars in 1087 bytes} [Fri Aug 5 19:37:47 2022] GET /identity/v3/services => generated 1497 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:37:47.424146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd74d5c-0389-4c81-9ce7-1470f409e172 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:47.466333 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7225e1c-a32b-459d-b1b6-e69543d82255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:47.467200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7225e1c-a32b-459d-b1b6-e69543d82255 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:47.467584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7225e1c-a32b-459d-b1b6-e69543d82255 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:47.467957 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7225e1c-a32b-459d-b1b6-e69543d82255 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:47.468716 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7225e1c-a32b-459d-b1b6-e69543d82255 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:47.470368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7225e1c-a32b-459d-b1b6-e69543d82255 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-a7225e1c-a32b-459d-b1b6-e69543d82255 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:47.481270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:47.482420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:37:47.483824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:47.485914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 475/928] 10.222.0.86 () {56 vars in 1132 bytes} [Fri Aug 5 19:37:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:47.493398 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b5132-d41d-416b-a88e-9a08aa0e7fab None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:47.525999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:47.630931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b5132-d41d-416b-a88e-9a08aa0e7fab None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:47.679001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8c169-4ff8-46dd-8273-b7f46fa2afc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:47.679627 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8c169-4ff8-46dd-8273-b7f46fa2afc6 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:47.679906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8c169-4ff8-46dd-8273-b7f46fa2afc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:47.680175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8c169-4ff8-46dd-8273-b7f46fa2afc6 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:47.680859 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8c169-4ff8-46dd-8273-b7f46fa2afc6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:47.681160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8c169-4ff8-46dd-8273-b7f46fa2afc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:47.682539 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8c169-4ff8-46dd-8273-b7f46fa2afc6 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:47.688435 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 454/929] 10.222.0.86 () {56 vars in 1126 bytes} [Fri Aug 5 19:37:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:47.694857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7225e1c-a32b-459d-b1b6-e69543d82255 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:47.748933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:47.809236 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7225e1c-a32b-459d-b1b6-e69543d82255 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:47.852514 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605377c-e649-4885-b8e0-2cf8d79937a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:47.853147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605377c-e649-4885-b8e0-2cf8d79937a3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:47.853423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605377c-e649-4885-b8e0-2cf8d79937a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:47.853693 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605377c-e649-4885-b8e0-2cf8d79937a3 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:47.854383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605377c-e649-4885-b8e0-2cf8d79937a3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:47.854711 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605377c-e649-4885-b8e0-2cf8d79937a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:47.856300 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605377c-e649-4885-b8e0-2cf8d79937a3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:47.862215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 476/930] 10.222.0.86 () {56 vars in 1126 bytes} [Fri Aug 5 19:37:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:47.870239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8c169-4ff8-46dd-8273-b7f46fa2afc6 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:47.973334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:47.986571 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8c169-4ff8-46dd-8273-b7f46fa2afc6 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:48.024311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747770cf-5f9d-4366-87ee-688e0465cb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:48.024926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747770cf-5f9d-4366-87ee-688e0465cb15 None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:48.025207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747770cf-5f9d-4366-87ee-688e0465cb15 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:48.025476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747770cf-5f9d-4366-87ee-688e0465cb15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:48.026185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747770cf-5f9d-4366-87ee-688e0465cb15 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:48.027579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747770cf-5f9d-4366-87ee-688e0465cb15 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:48.056839 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 455/931] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:37:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:37:48.259033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa633b8f-47cf-46d5-83d0-dc1280ef792a None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:48.259243 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa633b8f-47cf-46d5-83d0-dc1280ef792a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:48.259520 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa633b8f-47cf-46d5-83d0-dc1280ef792a None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:48.261047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 477/932] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:48.266814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4101dd2-faf0-4568-b5fd-530b1713fd24 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:48.267104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4101dd2-faf0-4568-b5fd-530b1713fd24 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:48.267377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4101dd2-faf0-4568-b5fd-530b1713fd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:48.293433 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-d4101dd2-faf0-4568-b5fd-530b1713fd24 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:48.294642 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d4101dd2-faf0-4568-b5fd-530b1713fd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:48.385099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 456/933] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:48.411228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa633b8f-47cf-46d5-83d0-dc1280ef792a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:48.534158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa633b8f-47cf-46d5-83d0-dc1280ef792a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:48.545290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:48.571817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70910b8c-9865-4fcf-a723-b3f961c4e24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:48.572437 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70910b8c-9865-4fcf-a723-b3f961c4e24f None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:48.572716 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70910b8c-9865-4fcf-a723-b3f961c4e24f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:48.572989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70910b8c-9865-4fcf-a723-b3f961c4e24f None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:48.573680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70910b8c-9865-4fcf-a723-b3f961c4e24f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:48.575137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70910b8c-9865-4fcf-a723-b3f961c4e24f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:48.577730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: WARNING keystone.server.flask.common [None req-70910b8c-9865-4fcf-a723-b3f961c4e24f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 05 19:37:48.599535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.sql.core [None req-70910b8c-9865-4fcf-a723-b3f961c4e24f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 19:37:48.599535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 19:37:48.599535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [parameters: {'id': '799f8af804ea4fe1b4a759acbd54cd3c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 19:37:48.599535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87004) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 19:37:48.624988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-70910b8c-9865-4fcf-a723-b3f961c4e24f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 799f8af804ea4fe1b4a759acbd54cd3c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 799f8af804ea4fe1b4a759acbd54cd3c. Aug 05 19:37:48.624988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:48.624988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:37:48.624988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:37:48.624988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:37:48.624988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:37:48.624988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:37:48.624988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:37:48.624988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:37:48.624988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:37:48.624988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:37:48.624988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:37:48.624988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application result.read() Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 05 19:37:48.625875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application yield resource Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise self.value Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application session.commit() Aug 05 19:37:48.626872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self.session.flush() Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application compat.raise_( Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 19:37:48.627801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 19:37:48.628709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application result.read() Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:37:48.629584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [parameters: {'id': '799f8af804ea4fe1b4a759acbd54cd3c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:48.632692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:48.634950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:48.637174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:48.637174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 05 19:37:48.637174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 05 19:37:48.637174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:48.637174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:48.637174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 05 19:37:48.637174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 05 19:37:48.637174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 799f8af804ea4fe1b4a759acbd54cd3c. Aug 05 19:37:48.637174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:48.646748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 478/934] 10.222.0.86 () {60 vars in 1140 bytes} [Fri Aug 5 19:37:48 2022] POST /identity/v3/projects => generated 262 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:48.649089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4101dd2-faf0-4568-b5fd-530b1713fd24 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:48.735106 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:48.789655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4101dd2-faf0-4568-b5fd-530b1713fd24 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:48.852218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53d79a-32cb-4491-b357-a2aeab126dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:48.853259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53d79a-32cb-4491-b357-a2aeab126dd0 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:48.853741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53d79a-32cb-4491-b357-a2aeab126dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:48.854216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53d79a-32cb-4491-b357-a2aeab126dd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:48.859615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce28b4a-85d8-450c-8390-fe565ea895ee None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:48.860180 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce28b4a-85d8-450c-8390-fe565ea895ee None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:48.860846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce28b4a-85d8-450c-8390-fe565ea895ee None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:48.867529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53d79a-32cb-4491-b357-a2aeab126dd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:48.869986 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53d79a-32cb-4491-b357-a2aeab126dd0 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:48.870527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 479/935] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:48 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:48.876266 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab9705-08e7-4401-a246-f99172b43bb0 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:48.876664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab9705-08e7-4401-a246-f99172b43bb0 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:48.877013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab9705-08e7-4401-a246-f99172b43bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-ab53d79a-32cb-4491-b357-a2aeab126dd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:48.883414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:48.884506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:48.885505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:48.885505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:48.885505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:48.885505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:48.885505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:48.885505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:48.885505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:48.885505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:48.885505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 457/936] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:48.891521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53d79a-32cb-4491-b357-a2aeab126dd0 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:48.903590 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-9dab9705-08e7-4401-a246-f99172b43bb0 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:48.905073 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9dab9705-08e7-4401-a246-f99172b43bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:49.007953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 480/937] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:49.030664 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53d79a-32cb-4491-b357-a2aeab126dd0 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:49.030846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab9705-08e7-4401-a246-f99172b43bb0 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:49.070305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:49.078688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f5637-6533-4c62-991a-f6047919d698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:49.079614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f5637-6533-4c62-991a-f6047919d698 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:49.080490 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f5637-6533-4c62-991a-f6047919d698 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:49.081220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f5637-6533-4c62-991a-f6047919d698 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:49.082121 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f5637-6533-4c62-991a-f6047919d698 None admin] RBAC: Adding query filter params (name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:49.082892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f5637-6533-4c62-991a-f6047919d698 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:49.085296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f5637-6533-4c62-991a-f6047919d698 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:49.109574 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 458/938] 10.222.0.86 () {56 vars in 1112 bytes} [Fri Aug 5 19:37:48 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:49.144813 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:49.153832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab9705-08e7-4401-a246-f99172b43bb0 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:49.195494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c578c-a493-4bcf-be33-604e39d2b2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:49.196527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c578c-a493-4bcf-be33-604e39d2b2c9 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:49.197043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c578c-a493-4bcf-be33-604e39d2b2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:49.197501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c578c-a493-4bcf-be33-604e39d2b2c9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:49.210916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c578c-a493-4bcf-be33-604e39d2b2c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-1c2c578c-a493-4bcf-be33-604e39d2b2c9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:49.216250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:49.217216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:49.218046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:49.218046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:49.218046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:49.218046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:49.218046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:49.218046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:49.218046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:49.218046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:49.218046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:49.218046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:49.218046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:37:49.218046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:49.220557 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 481/939] 10.222.0.86 () {56 vars in 1120 bytes} [Fri Aug 5 19:37:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:49.227611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f5637-6533-4c62-991a-f6047919d698 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:49.339206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f5637-6533-4c62-991a-f6047919d698 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:49.383367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08404e17-6efe-4cf5-bc2d-5c1a221c70f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:49.384245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08404e17-6efe-4cf5-bc2d-5c1a221c70f5 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:49.384525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08404e17-6efe-4cf5-bc2d-5c1a221c70f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:49.384861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08404e17-6efe-4cf5-bc2d-5c1a221c70f5 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:49.385586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08404e17-6efe-4cf5-bc2d-5c1a221c70f5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:49.385887 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08404e17-6efe-4cf5-bc2d-5c1a221c70f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:49.387290 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08404e17-6efe-4cf5-bc2d-5c1a221c70f5 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:49.397027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 459/940] 10.222.0.86 () {56 vars in 1115 bytes} [Fri Aug 5 19:37:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:49.402972 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c578c-a493-4bcf-be33-604e39d2b2c9 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:49.510891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c578c-a493-4bcf-be33-604e39d2b2c9 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:49.546653 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcadcf-ab83-420a-bc21-57da686b9aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:49.547303 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcadcf-ab83-420a-bc21-57da686b9aed None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:49.547579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcadcf-ab83-420a-bc21-57da686b9aed None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:49.547849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcadcf-ab83-420a-bc21-57da686b9aed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:49.558265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcadcf-ab83-420a-bc21-57da686b9aed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:49.559900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcadcf-ab83-420a-bc21-57da686b9aed None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-52dcadcf-ab83-420a-bc21-57da686b9aed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:49.576849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:49.577808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:49.578795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:49.578795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:49.578795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:49.578795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:49.578795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 482/941] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:49.584474 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08404e17-6efe-4cf5-bc2d-5c1a221c70f5 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:49.648879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:49.693539 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08404e17-6efe-4cf5-bc2d-5c1a221c70f5 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:49.727927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6e5b5-087b-4c2b-a9b6-7371e22cb0d6 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:49.728635 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6e5b5-087b-4c2b-a9b6-7371e22cb0d6 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:49.729180 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6e5b5-087b-4c2b-a9b6-7371e22cb0d6 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:49.731180 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 483/942] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:37:49.731606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a9437-b32e-41c8-9b7e-1bea46a2e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:49.732569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a9437-b32e-41c8-9b7e-1bea46a2e490 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:49.732868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a9437-b32e-41c8-9b7e-1bea46a2e490 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:49.733126 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a9437-b32e-41c8-9b7e-1bea46a2e490 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:49.733828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a9437-b32e-41c8-9b7e-1bea46a2e490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:49.734134 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a9437-b32e-41c8-9b7e-1bea46a2e490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:49.735600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a9437-b32e-41c8-9b7e-1bea46a2e490 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:49.736199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804d6a1-fa85-4f81-b89c-cc1b6e2fc2d2 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:49.736673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804d6a1-fa85-4f81-b89c-cc1b6e2fc2d2 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:49.737363 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804d6a1-fa85-4f81-b89c-cc1b6e2fc2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:49.754651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 460/943] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:49.762397 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a9437-b32e-41c8-9b7e-1bea46a2e490 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:49.765851 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-7804d6a1-fa85-4f81-b89c-cc1b6e2fc2d2 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:49.768284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7804d6a1-fa85-4f81-b89c-cc1b6e2fc2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:49.771367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:49.867834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 484/944] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:49.890873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804d6a1-fa85-4f81-b89c-cc1b6e2fc2d2 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:49.899768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a9437-b32e-41c8-9b7e-1bea46a2e490 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:49.917939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:49.950205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6b714-a058-4a5b-834b-675a7506688f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:49.950878 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6b714-a058-4a5b-834b-675a7506688f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:49.951160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6b714-a058-4a5b-834b-675a7506688f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:49.951434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6b714-a058-4a5b-834b-675a7506688f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:49.961895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6b714-a058-4a5b-834b-675a7506688f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:49.963659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6b714-a058-4a5b-834b-675a7506688f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-63d6b714-a058-4a5b-834b-675a7506688f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:49.980262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:49.981283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:49.982250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:49.982250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:49.982250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:49.982250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:49.982250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:49.982250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:49.982250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:49.982250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:49.982250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:37:49.982250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:49.982250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 461/945] 10.222.0.86 () {56 vars in 1117 bytes} [Fri Aug 5 19:37:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:49.989126 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6b714-a058-4a5b-834b-675a7506688f None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:50.019220 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804d6a1-fa85-4f81-b89c-cc1b6e2fc2d2 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:50.055598 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64f586-116b-4d29-8d7a-31c773b92e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:50.056348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64f586-116b-4d29-8d7a-31c773b92e10 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:50.056530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64f586-116b-4d29-8d7a-31c773b92e10 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:50.056788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64f586-116b-4d29-8d7a-31c773b92e10 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:50.057418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64f586-116b-4d29-8d7a-31c773b92e10 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:50.058888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64f586-116b-4d29-8d7a-31c773b92e10 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-4e64f586-116b-4d29-8d7a-31c773b92e10 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:50.069147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:50.070149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:50.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:50.074726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 485/946] 10.222.0.86 () {56 vars in 1126 bytes} [Fri Aug 5 19:37:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:50.076765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64f586-116b-4d29-8d7a-31c773b92e10 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:50.079251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:50.114275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6b714-a058-4a5b-834b-675a7506688f None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:50.152540 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e33853-50db-42df-8bdb-9c262f4edddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:50.153673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e33853-50db-42df-8bdb-9c262f4edddb None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:50.154178 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e33853-50db-42df-8bdb-9c262f4edddb None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:50.154732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e33853-50db-42df-8bdb-9c262f4edddb None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:50.155720 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e33853-50db-42df-8bdb-9c262f4edddb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:50.156225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e33853-50db-42df-8bdb-9c262f4edddb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:50.157842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e33853-50db-42df-8bdb-9c262f4edddb None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:50.181624 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 462/947] 10.222.0.86 () {56 vars in 1149 bytes} [Fri Aug 5 19:37:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:50.183052 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e33853-50db-42df-8bdb-9c262f4edddb None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:50.185939 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64f586-116b-4d29-8d7a-31c773b92e10 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:50.217438 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:50.223620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b3769-3ea7-431e-b562-378152559a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:50.225840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b3769-3ea7-431e-b562-378152559a18 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:50.226120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b3769-3ea7-431e-b562-378152559a18 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:50.226395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b3769-3ea7-431e-b562-378152559a18 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:50.227165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b3769-3ea7-431e-b562-378152559a18 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:50.227473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b3769-3ea7-431e-b562-378152559a18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:50.228872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b3769-3ea7-431e-b562-378152559a18 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:50.239069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 486/948] 10.222.0.86 () {56 vars in 1122 bytes} [Fri Aug 5 19:37:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:50.241238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b3769-3ea7-431e-b562-378152559a18 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:50.299503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e33853-50db-42df-8bdb-9c262f4edddb None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:50.314056 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:50.339862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e68bf-1dc7-481d-acaa-92fb6cf564ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:50.340439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e68bf-1dc7-481d-acaa-92fb6cf564ef None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:50.340716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e68bf-1dc7-481d-acaa-92fb6cf564ef None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:50.341010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e68bf-1dc7-481d-acaa-92fb6cf564ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:50.351952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b3769-3ea7-431e-b562-378152559a18 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:50.352125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e68bf-1dc7-481d-acaa-92fb6cf564ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:50.353534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e68bf-1dc7-481d-acaa-92fb6cf564ef None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-bb8e68bf-1dc7-481d-acaa-92fb6cf564ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:50.369544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:50.370488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:50.371423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:50.371423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:50.371423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:50.371423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:50.371423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 463/949] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:50.372139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b389917-46a2-45e1-ab06-3df9be0b5b83 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:50.372459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b389917-46a2-45e1-ab06-3df9be0b5b83 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:50.372641 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b389917-46a2-45e1-ab06-3df9be0b5b83 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:50.374149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 464/950] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:50.376477 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b389917-46a2-45e1-ab06-3df9be0b5b83 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:50.397776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a238e43-0599-47b5-a4d4-fee67c203270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:50.398399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a238e43-0599-47b5-a4d4-fee67c203270 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:50.398722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a238e43-0599-47b5-a4d4-fee67c203270 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:50.398997 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a238e43-0599-47b5-a4d4-fee67c203270 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:50.399690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a238e43-0599-47b5-a4d4-fee67c203270 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:50.399998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a238e43-0599-47b5-a4d4-fee67c203270 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:50.401369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a238e43-0599-47b5-a4d4-fee67c203270 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:50.406941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 487/951] 10.222.0.86 () {56 vars in 1122 bytes} [Fri Aug 5 19:37:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:50.408111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd6a7f-f6f1-4c67-b903-3600638e8115 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:50.408395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd6a7f-f6f1-4c67-b903-3600638e8115 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:50.408666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd6a7f-f6f1-4c67-b903-3600638e8115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:50.428178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-1dbd6a7f-f6f1-4c67-b903-3600638e8115 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:50.429472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1dbd6a7f-f6f1-4c67-b903-3600638e8115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:50.476530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:50.498930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b389917-46a2-45e1-ab06-3df9be0b5b83 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:50.510618 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 488/952] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:50.511292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd6a7f-f6f1-4c67-b903-3600638e8115 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:50.548503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2b789-c2c5-44c9-9e89-d29e3e64a553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:50.549114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2b789-c2c5-44c9-9e89-d29e3e64a553 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:50.549392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2b789-c2c5-44c9-9e89-d29e3e64a553 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:50.549662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2b789-c2c5-44c9-9e89-d29e3e64a553 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:50.550348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2b789-c2c5-44c9-9e89-d29e3e64a553 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:50.550680 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2b789-c2c5-44c9-9e89-d29e3e64a553 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:50.552667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2b789-c2c5-44c9-9e89-d29e3e64a553 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:50.564788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 465/953] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:50.565599 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2b789-c2c5-44c9-9e89-d29e3e64a553 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:50.581949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:50.630931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd6a7f-f6f1-4c67-b903-3600638e8115 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:50.672946 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405b50d-0843-44da-a925-26069b2b8041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:50.673648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405b50d-0843-44da-a925-26069b2b8041 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:50.673923 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405b50d-0843-44da-a925-26069b2b8041 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:50.674188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3405b50d-0843-44da-a925-26069b2b8041 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:50.674932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405b50d-0843-44da-a925-26069b2b8041 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:50.676341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3405b50d-0843-44da-a925-26069b2b8041 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:50.700709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2b789-c2c5-44c9-9e89-d29e3e64a553 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:50.706435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 489/954] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:37:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:37:50.707511 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405b50d-0843-44da-a925-26069b2b8041 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:50.763448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5556c7a-6c90-47e9-8445-558f4762c546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:50.764536 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5556c7a-6c90-47e9-8445-558f4762c546 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:50.765056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5556c7a-6c90-47e9-8445-558f4762c546 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:50.765535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5556c7a-6c90-47e9-8445-558f4762c546 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:50.766467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5556c7a-6c90-47e9-8445-558f4762c546 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:50.768162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5556c7a-6c90-47e9-8445-558f4762c546 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-e5556c7a-6c90-47e9-8445-558f4762c546 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:50.787221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:50.789522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:50.791109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 466/955] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:50.799475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5556c7a-6c90-47e9-8445-558f4762c546 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:50.804199 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:50.902677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3405b50d-0843-44da-a925-26069b2b8041 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:50.931833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:50.959727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b372c1-27aa-4a2c-829a-6681c20c8deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:50.960618 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b372c1-27aa-4a2c-829a-6681c20c8deb None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:50.961073 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b372c1-27aa-4a2c-829a-6681c20c8deb None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:50.961509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b372c1-27aa-4a2c-829a-6681c20c8deb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:50.975472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b372c1-27aa-4a2c-829a-6681c20c8deb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:50.977780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b372c1-27aa-4a2c-829a-6681c20c8deb None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:50.986225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5556c7a-6c90-47e9-8445-558f4762c546 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-54b372c1-27aa-4a2c-829a-6681c20c8deb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:50.995463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:50.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:50.997476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:50.997476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:50.997476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:50.997476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:50.997476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:50.997476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:50.997476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:50.997476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:50.997476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 490/956] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:51.002556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b372c1-27aa-4a2c-829a-6681c20c8deb None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:51.065966 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba1753-af25-42d8-90e9-bf3dff10f807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:51.067638 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba1753-af25-42d8-90e9-bf3dff10f807 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:51.069503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba1753-af25-42d8-90e9-bf3dff10f807 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:51.070769 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba1753-af25-42d8-90e9-bf3dff10f807 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:51.072122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba1753-af25-42d8-90e9-bf3dff10f807 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:51.073099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba1753-af25-42d8-90e9-bf3dff10f807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:51.076906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba1753-af25-42d8-90e9-bf3dff10f807 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:51.086866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 467/957] 10.222.0.86 () {56 vars in 1112 bytes} [Fri Aug 5 19:37:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:51.095772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba1753-af25-42d8-90e9-bf3dff10f807 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:51.179775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b372c1-27aa-4a2c-829a-6681c20c8deb None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:51.208853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:51.227760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4aae2-5101-4d58-82e3-37cf6832ba10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:51.228882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4aae2-5101-4d58-82e3-37cf6832ba10 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:51.229261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4aae2-5101-4d58-82e3-37cf6832ba10 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:51.229749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4aae2-5101-4d58-82e3-37cf6832ba10 None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:51.230795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4aae2-5101-4d58-82e3-37cf6832ba10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:51.231748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4aae2-5101-4d58-82e3-37cf6832ba10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:51.232897 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4aae2-5101-4d58-82e3-37cf6832ba10 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:51.248433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 491/958] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:51.256058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4aae2-5101-4d58-82e3-37cf6832ba10 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:51.267329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba1753-af25-42d8-90e9-bf3dff10f807 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:51.311080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:51.325788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927714e2-0f6a-4c21-9a55-efa0554c1ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:51.326845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927714e2-0f6a-4c21-9a55-efa0554c1ad9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:51.327155 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927714e2-0f6a-4c21-9a55-efa0554c1ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:51.327440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927714e2-0f6a-4c21-9a55-efa0554c1ad9 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:51.328124 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927714e2-0f6a-4c21-9a55-efa0554c1ad9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:51.328417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927714e2-0f6a-4c21-9a55-efa0554c1ad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:51.329812 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927714e2-0f6a-4c21-9a55-efa0554c1ad9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:51.342879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 468/959] 10.222.0.86 () {56 vars in 1112 bytes} [Fri Aug 5 19:37:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:51.407453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78def35e-693a-423c-8d06-69b6320d7118 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:51.407764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78def35e-693a-423c-8d06-69b6320d7118 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:51.408033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78def35e-693a-423c-8d06-69b6320d7118 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:51.409733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 469/960] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:37:51.417649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81603ab3-b44f-41f0-9003-842b81face31 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:51.418308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81603ab3-b44f-41f0-9003-842b81face31 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:51.418849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81603ab3-b44f-41f0-9003-842b81face31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:51.426548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4aae2-5101-4d58-82e3-37cf6832ba10 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:51.448988 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-81603ab3-b44f-41f0-9003-842b81face31 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:51.449569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-81603ab3-b44f-41f0-9003-842b81face31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:51.493387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255aa5c8-674c-4565-83bc-d6b8b88c64ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:51.494318 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255aa5c8-674c-4565-83bc-d6b8b88c64ac None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:51.494902 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255aa5c8-674c-4565-83bc-d6b8b88c64ac None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:51.495293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255aa5c8-674c-4565-83bc-d6b8b88c64ac None admin] PATH_INFO: `/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/7fc4868f25364311a504224fcd2c5a73/roles/6b65af6de416463283285acc9f0f40f9` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:51.498112 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255aa5c8-674c-4565-83bc-d6b8b88c64ac None admin] RBAC: Authorizing `identity:create_grant(project_id=9c73d2cfaa1c47829ca6936186d2bb26, user_id=7fc4868f25364311a504224fcd2c5a73, role_id=6b65af6de416463283285acc9f0f40f9)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:51.499903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255aa5c8-674c-4565-83bc-d6b8b88c64ac None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:51.530863 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 492/961] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:37:51 2022] PUT /identity/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/7fc4868f25364311a504224fcd2c5a73/roles/6b65af6de416463283285acc9f0f40f9 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:37:51.579593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 470/962] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:37:51.606825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255aa5c8-674c-4565-83bc-d6b8b88c64ac None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:51.736516 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:51.746086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255aa5c8-674c-4565-83bc-d6b8b88c64ac None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:51.789648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc7a5b-43ae-4bba-85ad-ea58ca6d163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:51.790512 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc7a5b-43ae-4bba-85ad-ea58ca6d163b None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:51.790942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc7a5b-43ae-4bba-85ad-ea58ca6d163b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:51.791350 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc7a5b-43ae-4bba-85ad-ea58ca6d163b None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:51.792218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc7a5b-43ae-4bba-85ad-ea58ca6d163b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:51.793807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc7a5b-43ae-4bba-85ad-ea58ca6d163b None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:51.813395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.sql.core [None req-9ccc7a5b-43ae-4bba-85ad-ea58ca6d163b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 19:37:51.813395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 19:37:51.813395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [parameters: {'user_id': '9a15f05d24244268aa2f83c08b46aeb5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 19:37:51.813395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87004) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-9ccc7a5b-43ae-4bba-85ad-ea58ca6d163b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application result.read() Aug 05 19:37:51.822855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:37:51.824084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application yield resource Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise self.value Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application session.commit() Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 19:37:51.825238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self.session.flush() Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application compat.raise_( Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 19:37:51.826344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:37:51.827340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application result.read() Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:37:51.828375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [parameters: {'user_id': '9a15f05d24244268aa2f83c08b46aeb5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:51.829383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 05 19:37:51.830175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 05 19:37:51.831113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 05 19:37:51.831113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 05 19:37:51.831113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 19:37:51.831113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 19:37:51.831113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 19:37:51.831113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:51.833866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 493/963] 10.222.0.86 () {60 vars in 1131 bytes} [Fri Aug 5 19:37:51 2022] POST /identity/v3/users => generated 160 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:51.840278 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81603ab3-b44f-41f0-9003-842b81face31 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:51.919553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:51.963886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81603ab3-b44f-41f0-9003-842b81face31 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:52.002681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99b5a7-6456-4fbd-a934-0035c6a2cc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:52.003404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99b5a7-6456-4fbd-a934-0035c6a2cc5f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:52.003778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99b5a7-6456-4fbd-a934-0035c6a2cc5f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:52.004137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99b5a7-6456-4fbd-a934-0035c6a2cc5f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:52.013680 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99b5a7-6456-4fbd-a934-0035c6a2cc5f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:52.015342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99b5a7-6456-4fbd-a934-0035c6a2cc5f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-7f99b5a7-6456-4fbd-a934-0035c6a2cc5f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:52.030500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:52.031743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:52.032902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:52.032902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:52.032902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:52.032902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:52.032902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:52.032902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:52.032902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:52.032902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:52.032902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:37:52.032902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:52.034797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 471/964] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:37:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:52.041754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc7a5b-43ae-4bba-85ad-ea58ca6d163b None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:52.156560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc7a5b-43ae-4bba-85ad-ea58ca6d163b None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:52.195604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e25aa-5df7-43df-84fd-d1d8dafb3312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:52.196516 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e25aa-5df7-43df-84fd-d1d8dafb3312 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:52.196936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e25aa-5df7-43df-84fd-d1d8dafb3312 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:52.197347 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e25aa-5df7-43df-84fd-d1d8dafb3312 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:52.198328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e25aa-5df7-43df-84fd-d1d8dafb3312 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:52.198864 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e25aa-5df7-43df-84fd-d1d8dafb3312 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:52.200757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e25aa-5df7-43df-84fd-d1d8dafb3312 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:52.250128 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 494/965] 10.222.0.86 () {56 vars in 1101 bytes} [Fri Aug 5 19:37:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:52.975598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a8a2b-13b6-4d7d-8981-86e640988384 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:52.976129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a8a2b-13b6-4d7d-8981-86e640988384 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:52.976129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a8a2b-13b6-4d7d-8981-86e640988384 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:52.977687 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 472/966] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:52.983372 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a37dd-3b75-40ad-8ff8-75ab84686a3f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:52.983664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a37dd-3b75-40ad-8ff8-75ab84686a3f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:52.983932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a37dd-3b75-40ad-8ff8-75ab84686a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:53.009675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-ac0a37dd-3b75-40ad-8ff8-75ab84686a3f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:53.012090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ac0a37dd-3b75-40ad-8ff8-75ab84686a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:53.027748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:53.112744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 495/967] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:53.136638 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a8a2b-13b6-4d7d-8981-86e640988384 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:53.261805 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a8a2b-13b6-4d7d-8981-86e640988384 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:53.290334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:53.307835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0af10-53ba-4505-8498-72938734368a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:53.308478 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0af10-53ba-4505-8498-72938734368a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:53.308756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0af10-53ba-4505-8498-72938734368a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:53.309032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0af10-53ba-4505-8498-72938734368a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:53.309721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0af10-53ba-4505-8498-72938734368a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:53.311176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0af10-53ba-4505-8498-72938734368a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-20f0af10-53ba-4505-8498-72938734368a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:53.320895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:53.322011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:53.322899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:53.326986 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 473/968] 10.222.0.86 () {56 vars in 1126 bytes} [Fri Aug 5 19:37:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:53.328477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a37dd-3b75-40ad-8ff8-75ab84686a3f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:53.457807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a37dd-3b75-40ad-8ff8-75ab84686a3f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:53.496846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f45bc5-16f3-43b4-8d88-adcd87ee3de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:53.497479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f45bc5-16f3-43b4-8d88-adcd87ee3de9 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:53.497756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f45bc5-16f3-43b4-8d88-adcd87ee3de9 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:53.498026 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f45bc5-16f3-43b4-8d88-adcd87ee3de9 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:53.498726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f45bc5-16f3-43b4-8d88-adcd87ee3de9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:53.499030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f45bc5-16f3-43b4-8d88-adcd87ee3de9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:53.500395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f45bc5-16f3-43b4-8d88-adcd87ee3de9 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:53.511600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 496/969] 10.222.0.86 () {56 vars in 1122 bytes} [Fri Aug 5 19:37:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:53.513479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0af10-53ba-4505-8498-72938734368a None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:53.589068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3c70b-95b4-49ae-828c-4e10e0e8f4ee None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:53.589644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3c70b-95b4-49ae-828c-4e10e0e8f4ee None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:53.590278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3c70b-95b4-49ae-828c-4e10e0e8f4ee None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:53.592432 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 497/970] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:37:53.597624 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedece1c-e62f-4195-b72d-3cf4cb7b015c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:53.598125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedece1c-e62f-4195-b72d-3cf4cb7b015c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:53.598575 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedece1c-e62f-4195-b72d-3cf4cb7b015c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:53.625336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-dedece1c-e62f-4195-b72d-3cf4cb7b015c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:53.627158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-dedece1c-e62f-4195-b72d-3cf4cb7b015c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:53.627927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0af10-53ba-4505-8498-72938734368a None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:53.632308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:53.664490 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7331b5-c6b0-4bf7-b0fc-75a356389f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:53.665111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7331b5-c6b0-4bf7-b0fc-75a356389f4b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:53.665390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7331b5-c6b0-4bf7-b0fc-75a356389f4b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:53.665663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7331b5-c6b0-4bf7-b0fc-75a356389f4b None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:53.666344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7331b5-c6b0-4bf7-b0fc-75a356389f4b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:53.666665 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7331b5-c6b0-4bf7-b0fc-75a356389f4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:53.668043 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7331b5-c6b0-4bf7-b0fc-75a356389f4b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:53.673261 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 474/971] 10.222.0.86 () {56 vars in 1122 bytes} [Fri Aug 5 19:37:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:53.679886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7331b5-c6b0-4bf7-b0fc-75a356389f4b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:53.713311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 498/972] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:53.730003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc29a6-a43c-43e9-b8cc-f9c4262d4307 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:53.730298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc29a6-a43c-43e9-b8cc-f9c4262d4307 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:53.730971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc29a6-a43c-43e9-b8cc-f9c4262d4307 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:53.732489 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 499/973] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:37:53.733411 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:53.735622 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc29a6-a43c-43e9-b8cc-f9c4262d4307 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:53.788884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7331b5-c6b0-4bf7-b0fc-75a356389f4b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:53.824222 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94731d4a-f8f8-404c-876d-14e17564d305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:53.824867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94731d4a-f8f8-404c-876d-14e17564d305 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:53.825148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94731d4a-f8f8-404c-876d-14e17564d305 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:53.825418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94731d4a-f8f8-404c-876d-14e17564d305 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:53.826276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94731d4a-f8f8-404c-876d-14e17564d305 None admin] RBAC: Adding query filter params (interface=admin, service_id=a53f38790f6f4890a5505982c9812ce9, region_id=RegionOne) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:53.826586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94731d4a-f8f8-404c-876d-14e17564d305 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:53.827991 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94731d4a-f8f8-404c-876d-14e17564d305 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:53.834281 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 475/974] 10.222.0.86 () {56 vars in 1249 bytes} [Fri Aug 5 19:37:53 2022] GET /identity/v3/endpoints?service_id=a53f38790f6f4890a5505982c9812ce9&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:53.835941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56da63-184c-4b6d-9520-2470eb88dcf6 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:53.836226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56da63-184c-4b6d-9520-2470eb88dcf6 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:53.836504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56da63-184c-4b6d-9520-2470eb88dcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:53.843936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc29a6-a43c-43e9-b8cc-f9c4262d4307 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:53.855688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-ce56da63-184c-4b6d-9520-2470eb88dcf6 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:53.856443 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ce56da63-184c-4b6d-9520-2470eb88dcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:53.879491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12f003-bf6a-453a-8fae-952a1907750a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:53.880753 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12f003-bf6a-453a-8fae-952a1907750a None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:53.881302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12f003-bf6a-453a-8fae-952a1907750a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:53.881302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12f003-bf6a-453a-8fae-952a1907750a None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:53.881831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12f003-bf6a-453a-8fae-952a1907750a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:53.883252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12f003-bf6a-453a-8fae-952a1907750a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:53.898302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:53.902754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 500/975] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:37:53 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:37:53.903970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12f003-bf6a-453a-8fae-952a1907750a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:53.941599 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 476/976] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:53.967193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56da63-184c-4b6d-9520-2470eb88dcf6 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:54.013029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:54.034799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12f003-bf6a-453a-8fae-952a1907750a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:54.081604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72598cda-bf7f-4890-8d73-399276fdd3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:54.081604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72598cda-bf7f-4890-8d73-399276fdd3ec None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:54.081873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72598cda-bf7f-4890-8d73-399276fdd3ec None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:54.082099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72598cda-bf7f-4890-8d73-399276fdd3ec None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:54.082715 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72598cda-bf7f-4890-8d73-399276fdd3ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:54.084979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72598cda-bf7f-4890-8d73-399276fdd3ec None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:54.091883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 501/977] 10.222.0.86 () {56 vars in 1087 bytes} [Fri Aug 5 19:37:53 2022] GET /identity/v3/services => generated 1727 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:37:54.104583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56da63-184c-4b6d-9520-2470eb88dcf6 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:54.146159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52578e-f794-4242-9468-f820e2c2f7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:54.147444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52578e-f794-4242-9468-f820e2c2f7e9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:54.148015 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52578e-f794-4242-9468-f820e2c2f7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:54.148507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52578e-f794-4242-9468-f820e2c2f7e9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:54.162418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52578e-f794-4242-9468-f820e2c2f7e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-8e52578e-f794-4242-9468-f820e2c2f7e9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:37:54.167949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:54.169032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:54.169902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:54.169902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:54.169902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:54.169902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:54.169902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:37:54.169902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:37:54.169902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:37:54.169902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:37:54.169902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:37:54.169902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:37:54.169902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:37:54.169902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:54.170583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 477/978] 10.222.0.86 () {56 vars in 1120 bytes} [Fri Aug 5 19:37:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:54.178472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72598cda-bf7f-4890-8d73-399276fdd3ec None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:54.253912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:54.299762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72598cda-bf7f-4890-8d73-399276fdd3ec None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:54.341805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef577a6-4f61-4ee3-9f43-af7285264a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:54.342734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef577a6-4f61-4ee3-9f43-af7285264a07 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:54.343021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef577a6-4f61-4ee3-9f43-af7285264a07 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:54.343292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef577a6-4f61-4ee3-9f43-af7285264a07 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:54.344013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef577a6-4f61-4ee3-9f43-af7285264a07 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:54.344315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef577a6-4f61-4ee3-9f43-af7285264a07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:54.345689 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef577a6-4f61-4ee3-9f43-af7285264a07 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:54.362610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 502/979] 10.222.0.86 () {56 vars in 1115 bytes} [Fri Aug 5 19:37:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:54.363950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52578e-f794-4242-9468-f820e2c2f7e9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:54.493823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:54.502716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52578e-f794-4242-9468-f820e2c2f7e9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:54.550160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9a02a-4f89-4428-bff0-3cc7aeadfe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:54.550825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9a02a-4f89-4428-bff0-3cc7aeadfe6e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:54.551103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9a02a-4f89-4428-bff0-3cc7aeadfe6e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:54.551370 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9a02a-4f89-4428-bff0-3cc7aeadfe6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:54.564815 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c9a02a-4f89-4428-bff0-3cc7aeadfe6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:54.566321 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c9a02a-4f89-4428-bff0-3cc7aeadfe6e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-17c9a02a-4f89-4428-bff0-3cc7aeadfe6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:54.583841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:54.584804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:54.585709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:54.585709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:54.585709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:54.585709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:54.586809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 478/980] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:54.591358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef577a6-4f61-4ee3-9f43-af7285264a07 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:54.736449 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef577a6-4f61-4ee3-9f43-af7285264a07 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:54.764945 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:54.776589 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b84fe2-60a6-429a-9715-67d3489e4caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:54.777210 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b84fe2-60a6-429a-9715-67d3489e4caa None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:54.777483 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b84fe2-60a6-429a-9715-67d3489e4caa None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:54.777749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b84fe2-60a6-429a-9715-67d3489e4caa None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:54.778607 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b84fe2-60a6-429a-9715-67d3489e4caa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:54.778936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b84fe2-60a6-429a-9715-67d3489e4caa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:54.780292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b84fe2-60a6-429a-9715-67d3489e4caa None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:54.795071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 503/981] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:54.800557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9a02a-4f89-4428-bff0-3cc7aeadfe6e None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:54.912272 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9a02a-4f89-4428-bff0-3cc7aeadfe6e None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:54.948832 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9872d83-73a8-4a68-a831-7731cec859ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:54.949475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9872d83-73a8-4a68-a831-7731cec859ee None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:54.949752 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9872d83-73a8-4a68-a831-7731cec859ee None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:54.950022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9872d83-73a8-4a68-a831-7731cec859ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:54.959366 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9872d83-73a8-4a68-a831-7731cec859ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:54.960826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9872d83-73a8-4a68-a831-7731cec859ee None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-e9872d83-73a8-4a68-a831-7731cec859ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:37:54.972757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:54.974308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:54.979671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:54.979671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:54.979671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:37:54.979671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:37:54.979671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:37:54.979671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:37:54.979671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:37:54.979671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:37:54.979671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:37:54.979671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:54.979671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 479/982] 10.222.0.86 () {56 vars in 1117 bytes} [Fri Aug 5 19:37:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:54.982398 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b84fe2-60a6-429a-9715-67d3489e4caa None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:55.092405 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b84fe2-60a6-429a-9715-67d3489e4caa None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:55.134794 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5820b76-aedb-4934-b57f-fd2f5bae73b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:55.135742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5820b76-aedb-4934-b57f-fd2f5bae73b3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:55.136183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5820b76-aedb-4934-b57f-fd2f5bae73b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:55.136612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5820b76-aedb-4934-b57f-fd2f5bae73b3 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:55.137975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5820b76-aedb-4934-b57f-fd2f5bae73b3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:55.138494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5820b76-aedb-4934-b57f-fd2f5bae73b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:55.140448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5820b76-aedb-4934-b57f-fd2f5bae73b3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:55.165269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 504/983] 10.222.0.86 () {56 vars in 1149 bytes} [Fri Aug 5 19:37:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:55.166784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9872d83-73a8-4a68-a831-7731cec859ee None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:55.180579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:55.296236 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9872d83-73a8-4a68-a831-7731cec859ee None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:55.333286 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a1d00-9696-4fdd-b6c9-326095831a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:55.333902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a1d00-9696-4fdd-b6c9-326095831a1f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:55.334891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a1d00-9696-4fdd-b6c9-326095831a1f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:55.334891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a1d00-9696-4fdd-b6c9-326095831a1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:55.411996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a1d00-9696-4fdd-b6c9-326095831a1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:55.414069 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a1d00-9696-4fdd-b6c9-326095831a1f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:55.428500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:55.547706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 480/984] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:37:55 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:37:55.549318 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5820b76-aedb-4934-b57f-fd2f5bae73b3 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:55.598290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:55.657867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5820b76-aedb-4934-b57f-fd2f5bae73b3 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:55.713468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560cf1f-2c89-4850-abcb-7e7952a23ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:55.714472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560cf1f-2c89-4850-abcb-7e7952a23ffc None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:55.714994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560cf1f-2c89-4850-abcb-7e7952a23ffc None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:55.715478 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560cf1f-2c89-4850-abcb-7e7952a23ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:55.734579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560cf1f-2c89-4850-abcb-7e7952a23ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:55.737345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560cf1f-2c89-4850-abcb-7e7952a23ffc None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-e560cf1f-2c89-4850-abcb-7e7952a23ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:37:55.769297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:55.771371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:55.772314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:37:55.772314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:37:55.772314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:37:55.772314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:55.772549 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 505/985] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:55.778688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a1d00-9696-4fdd-b6c9-326095831a1f None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:55.910463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a1d00-9696-4fdd-b6c9-326095831a1f None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:55.923954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:55.951336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc90bc3-c693-44f8-ab6e-9ed024964c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:55.951954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc90bc3-c693-44f8-ab6e-9ed024964c4c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:55.952293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc90bc3-c693-44f8-ab6e-9ed024964c4c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:55.952626 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc90bc3-c693-44f8-ab6e-9ed024964c4c None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:55.953383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc90bc3-c693-44f8-ab6e-9ed024964c4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:55.953751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc90bc3-c693-44f8-ab6e-9ed024964c4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:55.955207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc90bc3-c693-44f8-ab6e-9ed024964c4c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:55.968512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 481/986] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:55.973757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560cf1f-2c89-4850-abcb-7e7952a23ffc None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:56.018294 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a11c14-d8ab-40be-b277-e1b457735bc4 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:56.018676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a11c14-d8ab-40be-b277-e1b457735bc4 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:56.018877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a11c14-d8ab-40be-b277-e1b457735bc4 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:56.021047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 482/987] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:56.026513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fcf150-b7eb-4ba8-adb1-e68ec9f0a689 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:56.026843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fcf150-b7eb-4ba8-adb1-e68ec9f0a689 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:56.027104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fcf150-b7eb-4ba8-adb1-e68ec9f0a689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:56.049123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-d5fcf150-b7eb-4ba8-adb1-e68ec9f0a689 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:56.050446 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d5fcf150-b7eb-4ba8-adb1-e68ec9f0a689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:56.089093 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:56.096916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560cf1f-2c89-4850-abcb-7e7952a23ffc None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:56.137789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 483/988] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:37:56.150488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74801c36-5ba8-4049-a49e-2b650f205944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:56.151982 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74801c36-5ba8-4049-a49e-2b650f205944 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:56.152759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74801c36-5ba8-4049-a49e-2b650f205944 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:56.153458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74801c36-5ba8-4049-a49e-2b650f205944 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:56.162544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fcf150-b7eb-4ba8-adb1-e68ec9f0a689 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:56.167432 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74801c36-5ba8-4049-a49e-2b650f205944 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:56.169921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74801c36-5ba8-4049-a49e-2b650f205944 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-74801c36-5ba8-4049-a49e-2b650f205944 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:37:56.194749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:56.195745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:56.196666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:37:56.196666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:37:56.196666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:37:56.196666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:37:56.196666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:37:56.196666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:37:56.196666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:37:56.196666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:56.197103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 506/989] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:56.203246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74801c36-5ba8-4049-a49e-2b650f205944 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:56.276599 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:56.279479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fcf150-b7eb-4ba8-adb1-e68ec9f0a689 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:56.315691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf3ef3-ce78-4a6c-ae3b-517355bfcea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:56.316332 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf3ef3-ce78-4a6c-ae3b-517355bfcea3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:56.316435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74801c36-5ba8-4049-a49e-2b650f205944 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:56.316612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf3ef3-ce78-4a6c-ae3b-517355bfcea3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:56.316883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf3ef3-ce78-4a6c-ae3b-517355bfcea3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:56.318046 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf3ef3-ce78-4a6c-ae3b-517355bfcea3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:56.319657 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf3ef3-ce78-4a6c-ae3b-517355bfcea3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-78cf3ef3-ce78-4a6c-ae3b-517355bfcea3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:56.328777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:56.329897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:56.331132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 484/990] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:37:56.331973 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb33437-2c4e-49c7-a6c1-2c6e30ae1ea5 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:56.331973 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb33437-2c4e-49c7-a6c1-2c6e30ae1ea5 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:56.331973 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb33437-2c4e-49c7-a6c1-2c6e30ae1ea5 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:56.337961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 485/991] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:56.339162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af4e35-a131-4b32-8b58-fcd626613ee9 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:56.339858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af4e35-a131-4b32-8b58-fcd626613ee9 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:56.339858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af4e35-a131-4b32-8b58-fcd626613ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:56.353896 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17ddae-5dca-486d-958a-ec21338d7599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:56.354509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17ddae-5dca-486d-958a-ec21338d7599 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:56.354809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17ddae-5dca-486d-958a-ec21338d7599 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:56.355078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17ddae-5dca-486d-958a-ec21338d7599 None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:56.355818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17ddae-5dca-486d-958a-ec21338d7599 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:56.356115 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17ddae-5dca-486d-958a-ec21338d7599 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:56.357529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17ddae-5dca-486d-958a-ec21338d7599 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:56.358518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-03af4e35-a131-4b32-8b58-fcd626613ee9 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:56.359807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-03af4e35-a131-4b32-8b58-fcd626613ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:56.371613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 507/992] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:37:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:56.372548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17ddae-5dca-486d-958a-ec21338d7599 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:56.391673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:56.431623 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 486/993] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:37:56.432737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af4e35-a131-4b32-8b58-fcd626613ee9 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:56.479777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17ddae-5dca-486d-958a-ec21338d7599 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:56.518977 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c5125-791d-4eaf-bc40-0c46dbef7517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:56.519586 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c5125-791d-4eaf-bc40-0c46dbef7517 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:56.519860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c5125-791d-4eaf-bc40-0c46dbef7517 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:56.520126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c5125-791d-4eaf-bc40-0c46dbef7517 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:56.521118 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c5125-791d-4eaf-bc40-0c46dbef7517 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:56.521470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c5125-791d-4eaf-bc40-0c46dbef7517 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:56.522865 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542c5125-791d-4eaf-bc40-0c46dbef7517 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:56.527954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 508/994] 10.222.0.86 () {56 vars in 1112 bytes} [Fri Aug 5 19:37:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:56.528684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c5125-791d-4eaf-bc40-0c46dbef7517 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:56.543203 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:56.550541 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af4e35-a131-4b32-8b58-fcd626613ee9 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:56.596639 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609f856-77f7-4b6a-b7d6-13638d51471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:56.597263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609f856-77f7-4b6a-b7d6-13638d51471c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:56.597542 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609f856-77f7-4b6a-b7d6-13638d51471c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:56.597816 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609f856-77f7-4b6a-b7d6-13638d51471c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:56.599295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609f856-77f7-4b6a-b7d6-13638d51471c None admin] RBAC: Adding query filter params (role.id=6b65af6de416463283285acc9f0f40f9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=7fc4868f25364311a504224fcd2c5a73) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:56.599604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609f856-77f7-4b6a-b7d6-13638d51471c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:56.601051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609f856-77f7-4b6a-b7d6-13638d51471c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:56.607396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 487/995] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:37:56 2022] GET /identity/v3/role_assignments?user.id=7fc4868f25364311a504224fcd2c5a73&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=6b65af6de416463283285acc9f0f40f9 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:56.608186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609f856-77f7-4b6a-b7d6-13638d51471c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:56.627848 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:56.651467 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c5125-791d-4eaf-bc40-0c46dbef7517 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:56.692824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21174237-42c2-4b24-8147-509a1d9dab29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:56.693900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21174237-42c2-4b24-8147-509a1d9dab29 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:56.694299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21174237-42c2-4b24-8147-509a1d9dab29 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:56.694682 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21174237-42c2-4b24-8147-509a1d9dab29 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:56.695332 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21174237-42c2-4b24-8147-509a1d9dab29 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:56.697495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21174237-42c2-4b24-8147-509a1d9dab29 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-21174237-42c2-4b24-8147-509a1d9dab29 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:56.709628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:56.710764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:37:56.711890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 509/996] 10.222.0.86 () {56 vars in 1126 bytes} [Fri Aug 5 19:37:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:56.717102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21174237-42c2-4b24-8147-509a1d9dab29 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:56.732994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609f856-77f7-4b6a-b7d6-13638d51471c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:56.774149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4f548-c707-4b49-9390-9f5c14493900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:56.774803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4f548-c707-4b49-9390-9f5c14493900 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:56.775082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4f548-c707-4b49-9390-9f5c14493900 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:56.775351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4f548-c707-4b49-9390-9f5c14493900 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:56.776035 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4f548-c707-4b49-9390-9f5c14493900 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:56.776335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4f548-c707-4b49-9390-9f5c14493900 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:56.777705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4f548-c707-4b49-9390-9f5c14493900 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:56.784664 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 488/997] 10.222.0.86 () {56 vars in 1112 bytes} [Fri Aug 5 19:37:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:56.791287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4f548-c707-4b49-9390-9f5c14493900 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:56.820377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:56.844662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21174237-42c2-4b24-8147-509a1d9dab29 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:56.884079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd6cb1-a227-4066-aa9b-b250ca98698f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:56.884729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd6cb1-a227-4066-aa9b-b250ca98698f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:56.885007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd6cb1-a227-4066-aa9b-b250ca98698f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:56.885278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd6cb1-a227-4066-aa9b-b250ca98698f None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:56.885973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd6cb1-a227-4066-aa9b-b250ca98698f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:56.886274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd6cb1-a227-4066-aa9b-b250ca98698f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:56.887683 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd6cb1-a227-4066-aa9b-b250ca98698f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:56.892792 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 510/998] 10.222.0.86 () {56 vars in 1122 bytes} [Fri Aug 5 19:37:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:56.899632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd6cb1-a227-4066-aa9b-b250ca98698f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:56.911592 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:56.918998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4f548-c707-4b49-9390-9f5c14493900 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:56.965768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ca048-34d1-4fe3-8a5a-d47d93978652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:56.966781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ca048-34d1-4fe3-8a5a-d47d93978652 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:56.967233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ca048-34d1-4fe3-8a5a-d47d93978652 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:56.967669 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ca048-34d1-4fe3-8a5a-d47d93978652 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:56.968918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ca048-34d1-4fe3-8a5a-d47d93978652 None admin] RBAC: Adding query filter params (interface=public, service_id=66609265b3f74aa8bd6cf946f6bf5f6c, region_id=RegionOne) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:56.969409 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ca048-34d1-4fe3-8a5a-d47d93978652 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:56.971645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ca048-34d1-4fe3-8a5a-d47d93978652 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:56.984808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 489/999] 10.222.0.86 () {56 vars in 1251 bytes} [Fri Aug 5 19:37:56 2022] GET /identity/v3/endpoints?service_id=66609265b3f74aa8bd6cf946f6bf5f6c&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:56.986016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ca048-34d1-4fe3-8a5a-d47d93978652 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:57.029274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd6cb1-a227-4066-aa9b-b250ca98698f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:57.072907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1e65e-d9cc-4d71-ac12-87ae511bf15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:57.073444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1e65e-d9cc-4d71-ac12-87ae511bf15a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:57.073887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1e65e-d9cc-4d71-ac12-87ae511bf15a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:57.074161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1e65e-d9cc-4d71-ac12-87ae511bf15a None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:57.075262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c1e65e-d9cc-4d71-ac12-87ae511bf15a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:57.075573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c1e65e-d9cc-4d71-ac12-87ae511bf15a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:57.077266 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c1e65e-d9cc-4d71-ac12-87ae511bf15a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:57.083355 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 511/1000] 10.222.0.86 () {56 vars in 1122 bytes} [Fri Aug 5 19:37:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:57.089085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1e65e-d9cc-4d71-ac12-87ae511bf15a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:57.109074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:57.130745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ca048-34d1-4fe3-8a5a-d47d93978652 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:57.184243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa372fae-200d-40a7-ae92-0dece9ef9623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:57.185463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372fae-200d-40a7-ae92-0dece9ef9623 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:57.185927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372fae-200d-40a7-ae92-0dece9ef9623 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:57.186396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa372fae-200d-40a7-ae92-0dece9ef9623 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:57.187614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa372fae-200d-40a7-ae92-0dece9ef9623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:57.189144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:57.190431 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa372fae-200d-40a7-ae92-0dece9ef9623 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:57.208646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 490/1001] 10.222.0.86 () {56 vars in 1087 bytes} [Fri Aug 5 19:37:56 2022] GET /identity/v3/services => generated 1727 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:37:57.227936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1e65e-d9cc-4d71-ac12-87ae511bf15a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:57.266492 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cd43f-9d54-41bc-8124-261b1009f332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:57.267162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cd43f-9d54-41bc-8124-261b1009f332 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:57.267444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cd43f-9d54-41bc-8124-261b1009f332 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:57.267718 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cd43f-9d54-41bc-8124-261b1009f332 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:57.268444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cd43f-9d54-41bc-8124-261b1009f332 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:57.269840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cd43f-9d54-41bc-8124-261b1009f332 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:57.293328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 512/1002] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:37:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:37:58.882141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b81ea-a405-4e07-a8b2-089f3024d92a None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:58.882702 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b81ea-a405-4e07-a8b2-089f3024d92a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:58.882702 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b81ea-a405-4e07-a8b2-089f3024d92a None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:58.884081 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 491/1003] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:58.890564 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae62c0-08e8-40ff-a37a-af2cb069637c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:58.891151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae62c0-08e8-40ff-a37a-af2cb069637c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:58.891347 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae62c0-08e8-40ff-a37a-af2cb069637c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:58.914078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-a6ae62c0-08e8-40ff-a37a-af2cb069637c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:58.915327 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a6ae62c0-08e8-40ff-a37a-af2cb069637c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:59.009439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 513/1004] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:59.032265 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b81ea-a405-4e07-a8b2-089f3024d92a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:59.193059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b81ea-a405-4e07-a8b2-089f3024d92a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:59.235804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316dd0f-bcc9-4576-af74-a2536fb28e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:59.236755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316dd0f-bcc9-4576-af74-a2536fb28e59 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:59.237172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316dd0f-bcc9-4576-af74-a2536fb28e59 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:59.237588 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316dd0f-bcc9-4576-af74-a2536fb28e59 None admin] PATH_INFO: `/v3/services/image` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:59.238560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316dd0f-bcc9-4576-af74-a2536fb28e59 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:59.241358 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316dd0f-bcc9-4576-af74-a2536fb28e59 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-2316dd0f-bcc9-4576-af74-a2536fb28e59 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:59.255671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:59.256753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:37:59.257674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:59.262947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 492/1005] 10.222.0.86 () {56 vars in 1105 bytes} [Fri Aug 5 19:37:59 2022] GET /identity/v3/services/image => generated 86 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:59.264759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae62c0-08e8-40ff-a37a-af2cb069637c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:59.339772 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:59.350339 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e6166-5e38-4e07-8b7f-067f0511ae88 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:59.352811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e6166-5e38-4e07-8b7f-067f0511ae88 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:59.353319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e6166-5e38-4e07-8b7f-067f0511ae88 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:59.358690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 493/1006] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:59.361211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5318c83-1c8c-4a8c-a647-9d43fb85ec84 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:59.362429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5318c83-1c8c-4a8c-a647-9d43fb85ec84 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:59.362429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5318c83-1c8c-4a8c-a647-9d43fb85ec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:59.383165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-a5318c83-1c8c-4a8c-a647-9d43fb85ec84 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:59.384397 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a5318c83-1c8c-4a8c-a647-9d43fb85ec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:59.387508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae62c0-08e8-40ff-a37a-af2cb069637c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:59.394471 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:59.440198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9b75c-eed5-4e76-9941-0e21eeeb4b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:59.441115 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9b75c-eed5-4e76-9941-0e21eeeb4b92 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:59.441527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9b75c-eed5-4e76-9941-0e21eeeb4b92 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:59.441915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9b75c-eed5-4e76-9941-0e21eeeb4b92 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:59.443707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9b75c-eed5-4e76-9941-0e21eeeb4b92 None admin] RBAC: Adding query filter params (name=image) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:59.444179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9b75c-eed5-4e76-9941-0e21eeeb4b92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:59.446675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9b75c-eed5-4e76-9941-0e21eeeb4b92 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:59.455602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 514/1007] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:59.456852 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843fbed-52a2-4c1f-ad2b-f91cb92122a0 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:59.457134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843fbed-52a2-4c1f-ad2b-f91cb92122a0 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:59.457403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843fbed-52a2-4c1f-ad2b-f91cb92122a0 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:59.458960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 515/1008] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:37:59.461230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843fbed-52a2-4c1f-ad2b-f91cb92122a0 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:59.466667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 494/1009] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:59 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:37:59.466667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df98f-bf3a-4190-90e1-8bd8b96ba656 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:59.466667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df98f-bf3a-4190-90e1-8bd8b96ba656 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:59.466667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df98f-bf3a-4190-90e1-8bd8b96ba656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:59.491162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-f05df98f-bf3a-4190-90e1-8bd8b96ba656 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:37:59.492391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f05df98f-bf3a-4190-90e1-8bd8b96ba656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:37:59.564466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 495/1010] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:37:59 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:37:59.565318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05df98f-bf3a-4190-90e1-8bd8b96ba656 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:59.586282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843fbed-52a2-4c1f-ad2b-f91cb92122a0 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:59.626069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:59.639065 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcc46c-e732-46f6-b28d-78d026d2fbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:59.639687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcc46c-e732-46f6-b28d-78d026d2fbce None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:59.639960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcc46c-e732-46f6-b28d-78d026d2fbce None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:59.640226 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcc46c-e732-46f6-b28d-78d026d2fbce None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:59.640981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcc46c-e732-46f6-b28d-78d026d2fbce None admin] RBAC: Adding query filter params (type=image) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:59.641283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcc46c-e732-46f6-b28d-78d026d2fbce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:59.642674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcc46c-e732-46f6-b28d-78d026d2fbce None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:59.648196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 516/1011] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:59.651693 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcc46c-e732-46f6-b28d-78d026d2fbce None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:59.656885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:59.690252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05df98f-bf3a-4190-90e1-8bd8b96ba656 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:59.733865 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068fc3c-d014-4bce-8983-466418b6f8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:59.734492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068fc3c-d014-4bce-8983-466418b6f8bc None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:59.734788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068fc3c-d014-4bce-8983-466418b6f8bc None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:59.735063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068fc3c-d014-4bce-8983-466418b6f8bc None admin] PATH_INFO: `/v3/services/compute` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:59.735728 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068fc3c-d014-4bce-8983-466418b6f8bc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:59.737811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068fc3c-d014-4bce-8983-466418b6f8bc None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-4068fc3c-d014-4bce-8983-466418b6f8bc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:37:59.747764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:37:59.748837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:37:59.749718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 496/1012] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:37:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:37:59.755987 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068fc3c-d014-4bce-8983-466418b6f8bc None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:59.780033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcc46c-e732-46f6-b28d-78d026d2fbce None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:59.821270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87e390-ad8d-4025-a4a2-cd35903de3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:59.822052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87e390-ad8d-4025-a4a2-cd35903de3de None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:59.822399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87e390-ad8d-4025-a4a2-cd35903de3de None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:59.822857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87e390-ad8d-4025-a4a2-cd35903de3de None admin] PATH_INFO: `/v3/domains/default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:59.824140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87e390-ad8d-4025-a4a2-cd35903de3de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:59.825589 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87e390-ad8d-4025-a4a2-cd35903de3de None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:59.827596 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 517/1013] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:37:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:37:59.834924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87e390-ad8d-4025-a4a2-cd35903de3de None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:59.881107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068fc3c-d014-4bce-8983-466418b6f8bc None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:37:59.901114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:59.923508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a1623-dc56-43be-9855-f5945e116270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:37:59.924220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a1623-dc56-43be-9855-f5945e116270 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:37:59.924477 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a1623-dc56-43be-9855-f5945e116270 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:37:59.924751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a1623-dc56-43be-9855-f5945e116270 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:37:59.925521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a1623-dc56-43be-9855-f5945e116270 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:37:59.925790 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a1623-dc56-43be-9855-f5945e116270 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:37:59.927333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a1623-dc56-43be-9855-f5945e116270 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:37:59.932995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 497/1014] 10.222.0.86 () {56 vars in 1112 bytes} [Fri Aug 5 19:37:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:37:59.939466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a1623-dc56-43be-9855-f5945e116270 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:37:59.943588 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:37:59.960819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87e390-ad8d-4025-a4a2-cd35903de3de None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:00.005992 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b8bf5-cc7b-4ac6-8349-40cefd61f272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:00.006947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b8bf5-cc7b-4ac6-8349-40cefd61f272 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:00.007442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b8bf5-cc7b-4ac6-8349-40cefd61f272 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:00.007915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b8bf5-cc7b-4ac6-8349-40cefd61f272 None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:00.009310 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b8bf5-cc7b-4ac6-8349-40cefd61f272 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:00.010983 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b8bf5-cc7b-4ac6-8349-40cefd61f272 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:00.045296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 518/1015] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:37:59 2022] POST /identity/v3/projects => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:38:00.071644 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a1623-dc56-43be-9855-f5945e116270 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:00.115439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477cf58-7a37-4ea7-bc3b-01ed154036cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:00.116205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477cf58-7a37-4ea7-bc3b-01ed154036cb None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:00.116486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477cf58-7a37-4ea7-bc3b-01ed154036cb None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:00.116760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477cf58-7a37-4ea7-bc3b-01ed154036cb None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:00.117712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477cf58-7a37-4ea7-bc3b-01ed154036cb None admin] RBAC: Adding query filter params (type=compute) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:00.118167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477cf58-7a37-4ea7-bc3b-01ed154036cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:00.119681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477cf58-7a37-4ea7-bc3b-01ed154036cb None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:00.131119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 498/1016] 10.222.0.86 () {56 vars in 1112 bytes} [Fri Aug 5 19:37:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:00.132547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b8bf5-cc7b-4ac6-8349-40cefd61f272 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:00.263074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b8bf5-cc7b-4ac6-8349-40cefd61f272 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:00.271474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:00.300359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cccef7-409e-40aa-8c1a-cca9822954ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:00.301038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cccef7-409e-40aa-8c1a-cca9822954ea None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:00.301322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cccef7-409e-40aa-8c1a-cca9822954ea None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:00.302168 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cccef7-409e-40aa-8c1a-cca9822954ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:00.303107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cccef7-409e-40aa-8c1a-cca9822954ea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:00.304491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cccef7-409e-40aa-8c1a-cca9822954ea None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:00.329433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 519/1017] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:38:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:38:01.731796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe56a63-ac9f-4bd8-832e-0e55114ca807 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:01.732398 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe56a63-ac9f-4bd8-832e-0e55114ca807 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:01.732398 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe56a63-ac9f-4bd8-832e-0e55114ca807 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:01.733646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 499/1018] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:01.743814 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c7020-0d68-4996-a514-e1a12d1ab09d None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:01.743814 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c7020-0d68-4996-a514-e1a12d1ab09d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:01.743814 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c7020-0d68-4996-a514-e1a12d1ab09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:01.764355 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c04c7020-0d68-4996-a514-e1a12d1ab09d None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:01.765528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c04c7020-0d68-4996-a514-e1a12d1ab09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:01.850815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 520/1019] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:01 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:01.868758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe56a63-ac9f-4bd8-832e-0e55114ca807 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:01.954766 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:02.009453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe56a63-ac9f-4bd8-832e-0e55114ca807 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:02.044962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aadbd4-86a8-42b9-8a7d-bbb0fdaf078e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:02.045581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aadbd4-86a8-42b9-8a7d-bbb0fdaf078e None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:02.045857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aadbd4-86a8-42b9-8a7d-bbb0fdaf078e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:02.046129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aadbd4-86a8-42b9-8a7d-bbb0fdaf078e None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:02.046994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aadbd4-86a8-42b9-8a7d-bbb0fdaf078e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:02.048361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aadbd4-86a8-42b9-8a7d-bbb0fdaf078e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:02.070673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 500/1020] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:38:01 2022] POST /identity/v3/services => generated 242 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:38:02.174677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa1021-61f5-44b0-810b-47f04ec683f5 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:02.174970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa1021-61f5-44b0-810b-47f04ec683f5 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:02.175246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa1021-61f5-44b0-810b-47f04ec683f5 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:02.177414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 521/1021] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:02.183142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ddfe8-a527-4121-bd1b-9acacf196ef7 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:02.183462 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ddfe8-a527-4121-bd1b-9acacf196ef7 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:02.183706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ddfe8-a527-4121-bd1b-9acacf196ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:02.203625 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-1f3ddfe8-a527-4121-bd1b-9acacf196ef7 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:02.204853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-1f3ddfe8-a527-4121-bd1b-9acacf196ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:02.291489 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 501/1022] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:02.316507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa1021-61f5-44b0-810b-47f04ec683f5 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:02.420047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:02.437037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa1021-61f5-44b0-810b-47f04ec683f5 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:02.486182 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256ce7b-7d8a-404a-b5d7-7653c5a0dd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:02.487169 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256ce7b-7d8a-404a-b5d7-7653c5a0dd26 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:02.487627 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256ce7b-7d8a-404a-b5d7-7653c5a0dd26 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:02.488103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256ce7b-7d8a-404a-b5d7-7653c5a0dd26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:02.489556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256ce7b-7d8a-404a-b5d7-7653c5a0dd26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:02.491315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256ce7b-7d8a-404a-b5d7-7653c5a0dd26 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:02.493169 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 522/1023] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:02.499549 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ddfe8-a527-4121-bd1b-9acacf196ef7 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:02.554414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5e28a-ccb4-437e-add0-9f8c5b483416 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:02.554610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5e28a-ccb4-437e-add0-9f8c5b483416 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:02.554891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5e28a-ccb4-437e-add0-9f8c5b483416 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:02.556542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 523/1024] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:38:02.561640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf827518-9b1f-46f0-b011-500fec0ad74f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:02.561948 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf827518-9b1f-46f0-b011-500fec0ad74f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:02.562196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf827518-9b1f-46f0-b011-500fec0ad74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:02.570008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:02.584959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-bf827518-9b1f-46f0-b011-500fec0ad74f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:02.586481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-bf827518-9b1f-46f0-b011-500fec0ad74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:02.626193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ddfe8-a527-4121-bd1b-9acacf196ef7 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:02.671185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81259373-b1e9-4e24-abeb-aeae402ba2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:02.671811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81259373-b1e9-4e24-abeb-aeae402ba2a7 None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:02.672091 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81259373-b1e9-4e24-abeb-aeae402ba2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:02.672362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81259373-b1e9-4e24-abeb-aeae402ba2a7 None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:02.673038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 524/1025] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:38:02.673158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81259373-b1e9-4e24-abeb-aeae402ba2a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:02.674641 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81259373-b1e9-4e24-abeb-aeae402ba2a7 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:02.706658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 502/1026] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:38:02 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:38:02.706895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf827518-9b1f-46f0-b011-500fec0ad74f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:02.813981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:02.832310 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf827518-9b1f-46f0-b011-500fec0ad74f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:02.874448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5b996-9619-4031-aabf-4fb34d720e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:02.875179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5b996-9619-4031-aabf-4fb34d720e05 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:02.875451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5b996-9619-4031-aabf-4fb34d720e05 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:02.875717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5b996-9619-4031-aabf-4fb34d720e05 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:02.887337 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5b996-9619-4031-aabf-4fb34d720e05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-3fa5b996-9619-4031-aabf-4fb34d720e05 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:38:02.892188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:02.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:02.894218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:02.894218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:02.894218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:02.894218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:02.894218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:38:02.894218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:38:02.894218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:38:02.894218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:38:02.894218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:38:02.894218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:38:02.894218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:38:02.894218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:02.894218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 525/1027] 10.222.0.86 () {56 vars in 1120 bytes} [Fri Aug 5 19:38:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:38:02.900715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81259373-b1e9-4e24-abeb-aeae402ba2a7 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:03.028620 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81259373-b1e9-4e24-abeb-aeae402ba2a7 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:03.039068 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:03.068079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707a731-73af-4d87-8258-5b23c2a3dda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:03.068379 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707a731-73af-4d87-8258-5b23c2a3dda3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:03.068766 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707a731-73af-4d87-8258-5b23c2a3dda3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:03.068877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707a731-73af-4d87-8258-5b23c2a3dda3 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:03.069590 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707a731-73af-4d87-8258-5b23c2a3dda3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:03.069892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707a731-73af-4d87-8258-5b23c2a3dda3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:03.073318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707a731-73af-4d87-8258-5b23c2a3dda3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:03.084087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 503/1028] 10.222.0.86 () {56 vars in 1115 bytes} [Fri Aug 5 19:38:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:03.090190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5b996-9619-4031-aabf-4fb34d720e05 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:03.207196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5b996-9619-4031-aabf-4fb34d720e05 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:03.252837 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0498a-df54-436c-9305-cbfad332b31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:03.253472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0498a-df54-436c-9305-cbfad332b31f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:03.253744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0498a-df54-436c-9305-cbfad332b31f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:03.254010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0498a-df54-436c-9305-cbfad332b31f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:03.266534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0498a-df54-436c-9305-cbfad332b31f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:03.268318 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0498a-df54-436c-9305-cbfad332b31f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:03.282164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-55c0498a-df54-436c-9305-cbfad332b31f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:38:03.294495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:03.296103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:03.297761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:38:03.297761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:38:03.297761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:03.297761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:03.301673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707a731-73af-4d87-8258-5b23c2a3dda3 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:03.302323 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 526/1029] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:03.423945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707a731-73af-4d87-8258-5b23c2a3dda3 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:03.466644 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680bcdf-fd47-401e-a070-474927e92e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:03.467296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680bcdf-fd47-401e-a070-474927e92e6f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:03.467575 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680bcdf-fd47-401e-a070-474927e92e6f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:03.467825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680bcdf-fd47-401e-a070-474927e92e6f None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:03.468512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680bcdf-fd47-401e-a070-474927e92e6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:03.468808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680bcdf-fd47-401e-a070-474927e92e6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:03.470172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680bcdf-fd47-401e-a070-474927e92e6f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:03.485434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 504/1030] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:03.492594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0498a-df54-436c-9305-cbfad332b31f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:03.632058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0498a-df54-436c-9305-cbfad332b31f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:03.667049 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c8fee-1f7a-41d1-8f9d-3c162f39c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:03.667679 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c8fee-1f7a-41d1-8f9d-3c162f39c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:03.667959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c8fee-1f7a-41d1-8f9d-3c162f39c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:03.668232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c8fee-1f7a-41d1-8f9d-3c162f39c3d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:03.677681 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c8fee-1f7a-41d1-8f9d-3c162f39c3d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:03.679069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c8fee-1f7a-41d1-8f9d-3c162f39c3d9 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-e92c8fee-1f7a-41d1-8f9d-3c162f39c3d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:38:03.690481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:03.691609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:03.698161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:03.698161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:03.698161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:38:03.698161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:38:03.698161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:38:03.698161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:38:03.698161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:38:03.698161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:38:03.698161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:38:03.698161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:03.698161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 527/1031] 10.222.0.86 () {56 vars in 1093 bytes} [Fri Aug 5 19:38:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:03.698793 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680bcdf-fd47-401e-a070-474927e92e6f None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:03.728822 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:03.800972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680bcdf-fd47-401e-a070-474927e92e6f None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:03.836940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d72c7a-ba33-4e67-be7c-ce14daf28923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:03.837557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d72c7a-ba33-4e67-be7c-ce14daf28923 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:03.837837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d72c7a-ba33-4e67-be7c-ce14daf28923 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:03.838109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d72c7a-ba33-4e67-be7c-ce14daf28923 None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:03.838939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d72c7a-ba33-4e67-be7c-ce14daf28923 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:03.839247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d72c7a-ba33-4e67-be7c-ce14daf28923 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:03.840663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d72c7a-ba33-4e67-be7c-ce14daf28923 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:03.864693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 505/1032] 10.222.0.86 () {56 vars in 1133 bytes} [Fri Aug 5 19:38:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:03.866013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c8fee-1f7a-41d1-8f9d-3c162f39c3d9 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:03.933416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:03.980161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c8fee-1f7a-41d1-8f9d-3c162f39c3d9 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:04.015377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0fb21-e9bb-4a4e-92c8-42579086b8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:04.015993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0fb21-e9bb-4a4e-92c8-42579086b8cf None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:04.016268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0fb21-e9bb-4a4e-92c8-42579086b8cf None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:04.016537 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0fb21-e9bb-4a4e-92c8-42579086b8cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:04.087305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0fb21-e9bb-4a4e-92c8-42579086b8cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:04.088756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0fb21-e9bb-4a4e-92c8-42579086b8cf None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:04.146155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:04.195071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 528/1033] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:38:03 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:38:04.196627 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d72c7a-ba33-4e67-be7c-ce14daf28923 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:04.233910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e8b70-bfa4-471e-8921-d767343e8ede None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:04.234200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e8b70-bfa4-471e-8921-d767343e8ede None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:04.234474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e8b70-bfa4-471e-8921-d767343e8ede None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:04.235883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 529/1034] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:38:04.241457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1578d6-8d04-4b85-bc65-5d0ab6876e94 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:04.241743 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1578d6-8d04-4b85-bc65-5d0ab6876e94 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:04.242022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1578d6-8d04-4b85-bc65-5d0ab6876e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:04.260192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-1b1578d6-8d04-4b85-bc65-5d0ab6876e94 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:04.261373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1b1578d6-8d04-4b85-bc65-5d0ab6876e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:04.294592 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:04.307573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d72c7a-ba33-4e67-be7c-ce14daf28923 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:04.346729 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23940a-9521-43b0-a0cd-901662d4b76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:04.347466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23940a-9521-43b0-a0cd-901662d4b76a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:04.347746 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23940a-9521-43b0-a0cd-901662d4b76a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:04.348013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23940a-9521-43b0-a0cd-901662d4b76a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:04.355048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 530/1035] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:38:04.358974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23940a-9521-43b0-a0cd-901662d4b76a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:04.360416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23940a-9521-43b0-a0cd-901662d4b76a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:04.373462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1578d6-8d04-4b85-bc65-5d0ab6876e94 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-ec23940a-9521-43b0-a0cd-901662d4b76a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:38:04.378613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:04.380444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:04.381568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:38:04.381568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:38:04.381568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:04.381568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:04.381568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 506/1036] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:38:04.388170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23940a-9521-43b0-a0cd-901662d4b76a None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:04.482280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:04.503080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1578d6-8d04-4b85-bc65-5d0ab6876e94 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:04.514915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23940a-9521-43b0-a0cd-901662d4b76a None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:04.550144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07f075-1ee1-4339-8d8b-4c43eb5f3b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:04.550992 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07f075-1ee1-4339-8d8b-4c43eb5f3b57 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:04.551273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07f075-1ee1-4339-8d8b-4c43eb5f3b57 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:04.552057 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07f075-1ee1-4339-8d8b-4c43eb5f3b57 None admin] PATH_INFO: `/v3/services/image` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:04.552677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07f075-1ee1-4339-8d8b-4c43eb5f3b57 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:04.554850 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07f075-1ee1-4339-8d8b-4c43eb5f3b57 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:04.555482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99b784-080f-43e5-9982-e60443b3f7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:04.556276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99b784-080f-43e5-9982-e60443b3f7d0 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:04.556400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99b784-080f-43e5-9982-e60443b3f7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:04.556661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99b784-080f-43e5-9982-e60443b3f7d0 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:04.557364 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99b784-080f-43e5-9982-e60443b3f7d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:04.557676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99b784-080f-43e5-9982-e60443b3f7d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:04.559070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99b784-080f-43e5-9982-e60443b3f7d0 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-7c07f075-1ee1-4339-8d8b-4c43eb5f3b57 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:04.566028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:04.567890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:38:04.568838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:04.570713 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:04.571525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 531/1037] 10.222.0.86 () {56 vars in 1105 bytes} [Fri Aug 5 19:38:04 2022] GET /identity/v3/services/image => generated 86 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:38:04.577213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07f075-1ee1-4339-8d8b-4c43eb5f3b57 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:04.582217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 507/1038] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:04.584289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99b784-080f-43e5-9982-e60443b3f7d0 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:04.690309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99b784-080f-43e5-9982-e60443b3f7d0 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:04.694155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07f075-1ee1-4339-8d8b-4c43eb5f3b57 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:04.730965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100cf68-0fd7-410a-884b-b185d58b241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:04.731822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100cf68-0fd7-410a-884b-b185d58b241d None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:04.732030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100cf68-0fd7-410a-884b-b185d58b241d None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:04.732304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100cf68-0fd7-410a-884b-b185d58b241d None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:04.733004 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100cf68-0fd7-410a-884b-b185d58b241d None admin] RBAC: Adding query filter params (name=image) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:04.733312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100cf68-0fd7-410a-884b-b185d58b241d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:04.734969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100cf68-0fd7-410a-884b-b185d58b241d None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:04.741387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 532/1039] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:04.745936 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e484f51-82ec-4031-aae5-cb5866ea4c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:04.746383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e484f51-82ec-4031-aae5-cb5866ea4c0a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:04.746684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e484f51-82ec-4031-aae5-cb5866ea4c0a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:04.746848 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100cf68-0fd7-410a-884b-b185d58b241d None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:04.746960 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e484f51-82ec-4031-aae5-cb5866ea4c0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:04.758330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e484f51-82ec-4031-aae5-cb5866ea4c0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:04.759889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e484f51-82ec-4031-aae5-cb5866ea4c0a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:04.769887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-4e484f51-82ec-4031-aae5-cb5866ea4c0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:38:04.773719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:04.774700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:04.780479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:38:04.780479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:38:04.780479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:38:04.780479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:38:04.780479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:38:04.780479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:38:04.780479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:38:04.780479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:04.780479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 508/1040] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:38:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:38:04.780968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e484f51-82ec-4031-aae5-cb5866ea4c0a None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:04.849784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:04.870563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100cf68-0fd7-410a-884b-b185d58b241d None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:04.898868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e484f51-82ec-4031-aae5-cb5866ea4c0a None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:04.908537 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba981d9c-9437-417e-a31e-00813de3865d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:04.909159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba981d9c-9437-417e-a31e-00813de3865d None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:04.909435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba981d9c-9437-417e-a31e-00813de3865d None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:04.909705 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba981d9c-9437-417e-a31e-00813de3865d None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:04.910382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba981d9c-9437-417e-a31e-00813de3865d None admin] RBAC: Adding query filter params (type=image) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:04.910738 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba981d9c-9437-417e-a31e-00813de3865d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:04.912099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba981d9c-9437-417e-a31e-00813de3865d None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:04.917328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 533/1041] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:04.918493 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85765f1-8843-42af-b7fd-45b893605348 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:04.918804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85765f1-8843-42af-b7fd-45b893605348 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:04.919077 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85765f1-8843-42af-b7fd-45b893605348 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:04.920552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 534/1042] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:04.922943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85765f1-8843-42af-b7fd-45b893605348 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:04.935444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec09e2-37f2-4b7b-8693-9d5d812e095b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:04.936066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec09e2-37f2-4b7b-8693-9d5d812e095b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:04.936346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec09e2-37f2-4b7b-8693-9d5d812e095b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:04.936622 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec09e2-37f2-4b7b-8693-9d5d812e095b None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:04.937365 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec09e2-37f2-4b7b-8693-9d5d812e095b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:04.937699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec09e2-37f2-4b7b-8693-9d5d812e095b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:04.939407 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec09e2-37f2-4b7b-8693-9d5d812e095b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:04.952988 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 509/1043] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:38:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:04.954186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c49f-3614-4567-ab09-5f88c1bb08e9 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:04.954469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c49f-3614-4567-ab09-5f88c1bb08e9 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:04.954781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2c49f-3614-4567-ab09-5f88c1bb08e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:04.972960 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-32a2c49f-3614-4567-ab09-5f88c1bb08e9 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:04.974178 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-32a2c49f-3614-4567-ab09-5f88c1bb08e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:05.008989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:05.032490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85765f1-8843-42af-b7fd-45b893605348 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:05.047370 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 510/1044] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:05.048221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2c49f-3614-4567-ab09-5f88c1bb08e9 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:05.077131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cf2fe-2f19-4e53-bf4f-9e87d2b24d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:05.078009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf2fe-2f19-4e53-bf4f-9e87d2b24d54 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:05.078423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf2fe-2f19-4e53-bf4f-9e87d2b24d54 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:05.078824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf2fe-2f19-4e53-bf4f-9e87d2b24d54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:05.079891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cf2fe-2f19-4e53-bf4f-9e87d2b24d54 None admin] RBAC: Adding query filter params (interface=public, service_id=313d0140a5cb4ea1a77a959a615db8e8, region_id=RegionOne) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:05.080345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cf2fe-2f19-4e53-bf4f-9e87d2b24d54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:05.081966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cf2fe-2f19-4e53-bf4f-9e87d2b24d54 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:05.094742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 535/1045] 10.222.0.86 () {56 vars in 1251 bytes} [Fri Aug 5 19:38:04 2022] GET /identity/v3/endpoints?service_id=313d0140a5cb4ea1a77a959a615db8e8&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:05.094742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cf2fe-2f19-4e53-bf4f-9e87d2b24d54 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:05.100320 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:05.150351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2c49f-3614-4567-ab09-5f88c1bb08e9 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:05.184162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2be2d0-59fd-4ed3-9fe6-4524d8ed6a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:05.184762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2be2d0-59fd-4ed3-9fe6-4524d8ed6a28 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:05.185036 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2be2d0-59fd-4ed3-9fe6-4524d8ed6a28 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:05.185299 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2be2d0-59fd-4ed3-9fe6-4524d8ed6a28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:05.186068 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2be2d0-59fd-4ed3-9fe6-4524d8ed6a28 None admin] RBAC: Adding query filter params (role.id=6b65af6de416463283285acc9f0f40f9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=ab8f9d9cf4ca4e38ae6eb67e949b40fd) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:05.186369 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2be2d0-59fd-4ed3-9fe6-4524d8ed6a28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:05.188070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2be2d0-59fd-4ed3-9fe6-4524d8ed6a28 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:05.191442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cf2fe-2f19-4e53-bf4f-9e87d2b24d54 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:05.194292 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 511/1046] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:38:05 2022] GET /identity/v3/role_assignments?user.id=ab8f9d9cf4ca4e38ae6eb67e949b40fd&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=6b65af6de416463283285acc9f0f40f9 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:05.195100 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2be2d0-59fd-4ed3-9fe6-4524d8ed6a28 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:05.240647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f56fe-f26f-440b-ba3f-c4508ff2544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:05.241687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f56fe-f26f-440b-ba3f-c4508ff2544e None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:05.242209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f56fe-f26f-440b-ba3f-c4508ff2544e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:05.242481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f56fe-f26f-440b-ba3f-c4508ff2544e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:05.255614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f56fe-f26f-440b-ba3f-c4508ff2544e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-ae0f56fe-f26f-440b-ba3f-c4508ff2544e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:38:05.260984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:05.262137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:05.263132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:05.263132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:05.263132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:05.263132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:05.263132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:38:05.263132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:38:05.263132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:38:05.263132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:38:05.263132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:38:05.263132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:38:05.263132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:05.263132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:05.263132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 536/1047] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:38:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:05.268548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f56fe-f26f-440b-ba3f-c4508ff2544e None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:05.274983 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:05.322444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2be2d0-59fd-4ed3-9fe6-4524d8ed6a28 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:05.337911 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:05.366429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e62c91-af16-48a8-aac0-595229a6e819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:05.367500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e62c91-af16-48a8-aac0-595229a6e819 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:05.367886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e62c91-af16-48a8-aac0-595229a6e819 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:05.368343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e62c91-af16-48a8-aac0-595229a6e819 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:05.369294 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e62c91-af16-48a8-aac0-595229a6e819 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:05.371077 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e62c91-af16-48a8-aac0-595229a6e819 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:05.377888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 512/1048] 10.222.0.86 () {56 vars in 1087 bytes} [Fri Aug 5 19:38:05 2022] GET /identity/v3/services => generated 1957 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:38:05.387074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f56fe-f26f-440b-ba3f-c4508ff2544e None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:05.433616 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97dcf5-76ba-46aa-b380-589af5470482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:05.434550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97dcf5-76ba-46aa-b380-589af5470482 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:05.434998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97dcf5-76ba-46aa-b380-589af5470482 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:05.435406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97dcf5-76ba-46aa-b380-589af5470482 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:05.436293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97dcf5-76ba-46aa-b380-589af5470482 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:05.436776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97dcf5-76ba-46aa-b380-589af5470482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:05.438595 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97dcf5-76ba-46aa-b380-589af5470482 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:05.454799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 537/1049] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:38:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:05.456786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e62c91-af16-48a8-aac0-595229a6e819 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:05.574325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e62c91-af16-48a8-aac0-595229a6e819 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:05.613166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5a187-5a94-4acb-a561-c9766f3c6e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:05.613841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5a187-5a94-4acb-a561-c9766f3c6e02 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:05.614691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5a187-5a94-4acb-a561-c9766f3c6e02 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:05.614964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5a187-5a94-4acb-a561-c9766f3c6e02 None admin] PATH_INFO: `/v3/users/f3879f9dd02248e88c88e8c7e5238370` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:05.628878 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5a187-5a94-4acb-a561-c9766f3c6e02 None admin] RBAC: Authorizing `identity:get_user(user_id=f3879f9dd02248e88c88e8c7e5238370)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:05.630375 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5a187-5a94-4acb-a561-c9766f3c6e02 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:05.632132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 513/1050] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:05 2022] GET /identity/v3/users/f3879f9dd02248e88c88e8c7e5238370 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:05.638091 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97dcf5-76ba-46aa-b380-589af5470482 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:05.711979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:05.749866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97dcf5-76ba-46aa-b380-589af5470482 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:05.787155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3393ca-118c-4a5d-9632-988b045d49ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:05.787770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3393ca-118c-4a5d-9632-988b045d49ca None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:05.788038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3393ca-118c-4a5d-9632-988b045d49ca None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:05.788315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3393ca-118c-4a5d-9632-988b045d49ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:05.859189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3393ca-118c-4a5d-9632-988b045d49ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:05.860658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3393ca-118c-4a5d-9632-988b045d49ca None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:05.927776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:05.973433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 538/1051] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:38:05 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:38:05.976439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5a187-5a94-4acb-a561-c9766f3c6e02 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:06.014598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:06.080985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5a187-5a94-4acb-a561-c9766f3c6e02 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:06.116001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e83f0-43bc-4c2b-891d-82e1e78065ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:06.116549 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e83f0-43bc-4c2b-891d-82e1e78065ad None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:06.116825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e83f0-43bc-4c2b-891d-82e1e78065ad None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:06.117094 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e83f0-43bc-4c2b-891d-82e1e78065ad None admin] PATH_INFO: `/v3/projects/e1b4cd6da17d4492b9220460c4eed905` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:06.118245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e83f0-43bc-4c2b-891d-82e1e78065ad None admin] RBAC: Authorizing `identity:get_project(project_id=e1b4cd6da17d4492b9220460c4eed905)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:06.119705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e83f0-43bc-4c2b-891d-82e1e78065ad None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:06.126383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 514/1052] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:38:05 2022] GET /identity/v3/projects/e1b4cd6da17d4492b9220460c4eed905 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:06.127918 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3393ca-118c-4a5d-9632-988b045d49ca None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:06.235562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3393ca-118c-4a5d-9632-988b045d49ca None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:06.269726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aafc315-67c0-4fed-96bb-f051768ec591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:06.271151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafc315-67c0-4fed-96bb-f051768ec591 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:06.271423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafc315-67c0-4fed-96bb-f051768ec591 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:06.271691 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aafc315-67c0-4fed-96bb-f051768ec591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:06.272480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aafc315-67c0-4fed-96bb-f051768ec591 None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=e1b4cd6da17d4492b9220460c4eed905, user.id=f3879f9dd02248e88c88e8c7e5238370) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:06.272783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aafc315-67c0-4fed-96bb-f051768ec591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:06.274213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aafc315-67c0-4fed-96bb-f051768ec591 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:06.286681 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 539/1053] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:38:06 2022] GET /identity/v3/role_assignments?user.id=f3879f9dd02248e88c88e8c7e5238370&scope.project.id=e1b4cd6da17d4492b9220460c4eed905&role.id=93eb204498fc49d297bcd1841e00818d => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:07.239923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bc4cd-831f-48dd-b4a5-745fef91d2dd None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:07.240623 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bc4cd-831f-48dd-b4a5-745fef91d2dd None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:07.240623 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bc4cd-831f-48dd-b4a5-745fef91d2dd None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:07.241974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 515/1054] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:07.246948 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b4677-50af-415e-9948-062f1d31657b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:07.247360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b4677-50af-415e-9948-062f1d31657b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:07.247525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b4677-50af-415e-9948-062f1d31657b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:07.267707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-6c2b4677-50af-415e-9948-062f1d31657b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:07.268893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6c2b4677-50af-415e-9948-062f1d31657b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:07.295494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:07.343131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 540/1055] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:07.367115 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bc4cd-831f-48dd-b4a5-745fef91d2dd None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:07.431610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190b2dd-f66c-4631-a2be-ed78dc969034 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:07.431897 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190b2dd-f66c-4631-a2be-ed78dc969034 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:07.432169 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190b2dd-f66c-4631-a2be-ed78dc969034 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:07.433654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 541/1056] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:38:07.438507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ec77c-c6f6-4664-8e7e-c8bd7aee344f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:07.438811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ec77c-c6f6-4664-8e7e-c8bd7aee344f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:07.439102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ec77c-c6f6-4664-8e7e-c8bd7aee344f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:07.457435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-4e1ec77c-c6f6-4664-8e7e-c8bd7aee344f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:07.458672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4e1ec77c-c6f6-4664-8e7e-c8bd7aee344f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:07.484608 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bc4cd-831f-48dd-b4a5-745fef91d2dd None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:07.487415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:07.527700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c3a6f-d57f-4103-8a62-6f302e4582d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:07.528337 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c3a6f-d57f-4103-8a62-6f302e4582d1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:07.528611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c3a6f-d57f-4103-8a62-6f302e4582d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:07.528869 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c3a6f-d57f-4103-8a62-6f302e4582d1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:07.533903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 542/1057] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:38:07.540374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c3a6f-d57f-4103-8a62-6f302e4582d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-c06c3a6f-d57f-4103-8a62-6f302e4582d1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:38:07.545252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:07.546256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:07.547204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:07.547204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:07.547204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:07.547204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:07.547204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:38:07.547204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:38:07.547204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:38:07.547204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:38:07.547204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:38:07.547204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:38:07.547204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:38:07.547204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:07.547204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 516/1058] 10.222.0.86 () {56 vars in 1120 bytes} [Fri Aug 5 19:38:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:38:07.555949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ec77c-c6f6-4664-8e7e-c8bd7aee344f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:07.557118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c3a6f-d57f-4103-8a62-6f302e4582d1 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:07.652755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:07.664705 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ec77c-c6f6-4664-8e7e-c8bd7aee344f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:07.674453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c3a6f-d57f-4103-8a62-6f302e4582d1 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:07.701875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d42c69-40df-4ce5-90bb-d41f21535619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:07.702491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d42c69-40df-4ce5-90bb-d41f21535619 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:07.702793 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d42c69-40df-4ce5-90bb-d41f21535619 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:07.703086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d42c69-40df-4ce5-90bb-d41f21535619 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:07.703763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d42c69-40df-4ce5-90bb-d41f21535619 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:07.704064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d42c69-40df-4ce5-90bb-d41f21535619 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:07.705433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d42c69-40df-4ce5-90bb-d41f21535619 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:07.712551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8880f-400b-453f-baef-785bdd110d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:07.713389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8880f-400b-453f-baef-785bdd110d76 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:07.713808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8880f-400b-453f-baef-785bdd110d76 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:07.714139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 543/1059] 10.222.0.86 () {56 vars in 1115 bytes} [Fri Aug 5 19:38:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:07.714419 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8880f-400b-453f-baef-785bdd110d76 None admin] PATH_INFO: `/v3/services/image` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:07.715077 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8880f-400b-453f-baef-785bdd110d76 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:07.716502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8880f-400b-453f-baef-785bdd110d76 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:07.719715 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d42c69-40df-4ce5-90bb-d41f21535619 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-52f8880f-400b-453f-baef-785bdd110d76 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:07.728171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:07.729295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:07.730180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 517/1060] 10.222.0.86 () {56 vars in 1105 bytes} [Fri Aug 5 19:38:07 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:07.735061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8880f-400b-453f-baef-785bdd110d76 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:07.769074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:07.851823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8880f-400b-453f-baef-785bdd110d76 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:07.852765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d42c69-40df-4ce5-90bb-d41f21535619 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:07.887561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064fb05-6ec5-4135-8922-dcd6ee998c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:07.888174 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064fb05-6ec5-4135-8922-dcd6ee998c24 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:07.888451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064fb05-6ec5-4135-8922-dcd6ee998c24 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:07.888721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064fb05-6ec5-4135-8922-dcd6ee998c24 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:07.889389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064fb05-6ec5-4135-8922-dcd6ee998c24 None admin] RBAC: Adding query filter params (name=image) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:07.889690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064fb05-6ec5-4135-8922-dcd6ee998c24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:07.890308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d93770-e3fa-44b6-be13-8f791cfc4bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:07.891045 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d93770-e3fa-44b6-be13-8f791cfc4bec None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:07.891154 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064fb05-6ec5-4135-8922-dcd6ee998c24 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:07.891393 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d93770-e3fa-44b6-be13-8f791cfc4bec None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:07.891755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d93770-e3fa-44b6-be13-8f791cfc4bec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:07.897013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 518/1061] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:07.902371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064fb05-6ec5-4135-8922-dcd6ee998c24 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:07.903981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d93770-e3fa-44b6-be13-8f791cfc4bec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:07.905439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d93770-e3fa-44b6-be13-8f791cfc4bec None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-f6d93770-e3fa-44b6-be13-8f791cfc4bec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:38:07.921456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:07.922472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:07.923411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:38:07.923411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:38:07.923411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:07.923411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:07.923411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 544/1062] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:07.929414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d93770-e3fa-44b6-be13-8f791cfc4bec None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:07.933834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:08.013335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:08.024725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064fb05-6ec5-4135-8922-dcd6ee998c24 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:08.048560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d93770-e3fa-44b6-be13-8f791cfc4bec None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:08.059551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae13bb-ddc2-4852-ba3f-6cc450d5e807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:08.060169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae13bb-ddc2-4852-ba3f-6cc450d5e807 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:08.060445 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae13bb-ddc2-4852-ba3f-6cc450d5e807 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:08.060715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae13bb-ddc2-4852-ba3f-6cc450d5e807 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:08.061384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae13bb-ddc2-4852-ba3f-6cc450d5e807 None admin] RBAC: Adding query filter params (type=image) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:08.061691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae13bb-ddc2-4852-ba3f-6cc450d5e807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:08.063329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae13bb-ddc2-4852-ba3f-6cc450d5e807 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:08.068888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 519/1063] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:08.074241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae13bb-ddc2-4852-ba3f-6cc450d5e807 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:08.085734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4c9d2-a12f-4355-b8d1-18bb224ab282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:08.086353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4c9d2-a12f-4355-b8d1-18bb224ab282 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:08.086652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4c9d2-a12f-4355-b8d1-18bb224ab282 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:08.086931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4c9d2-a12f-4355-b8d1-18bb224ab282 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:08.087618 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4c9d2-a12f-4355-b8d1-18bb224ab282 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:08.087919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4c9d2-a12f-4355-b8d1-18bb224ab282 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:08.089280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4c9d2-a12f-4355-b8d1-18bb224ab282 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:08.106319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 545/1064] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:08.106974 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4c9d2-a12f-4355-b8d1-18bb224ab282 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:08.179315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:08.184810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae13bb-ddc2-4852-ba3f-6cc450d5e807 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:08.220386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c77d14-3b61-49b4-ad8c-c216902bcfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:08.220998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c77d14-3b61-49b4-ad8c-c216902bcfdf None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:08.221271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c77d14-3b61-49b4-ad8c-c216902bcfdf None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:08.221542 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c77d14-3b61-49b4-ad8c-c216902bcfdf None admin] PATH_INFO: `/v3/endpoints` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:08.222241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c77d14-3b61-49b4-ad8c-c216902bcfdf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:08.223812 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c77d14-3b61-49b4-ad8c-c216902bcfdf None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:08.226158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4c9d2-a12f-4355-b8d1-18bb224ab282 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:08.252124 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 520/1065] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:38:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:38:08.281173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdab44-51f8-4e74-81a2-9068ef36f032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:08.281815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdab44-51f8-4e74-81a2-9068ef36f032 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:08.282096 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdab44-51f8-4e74-81a2-9068ef36f032 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:08.282365 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdab44-51f8-4e74-81a2-9068ef36f032 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:08.291041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdab44-51f8-4e74-81a2-9068ef36f032 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:08.292513 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdab44-51f8-4e74-81a2-9068ef36f032 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-cbbdab44-51f8-4e74-81a2-9068ef36f032 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:38:08.306280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:08.307370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:08.308825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:08.308825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:08.308825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:38:08.308825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:38:08.308825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:38:08.308825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:38:08.308825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:38:08.308825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:38:08.308825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:38:08.308825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:08.308825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 546/1066] 10.222.0.86 () {56 vars in 1093 bytes} [Fri Aug 5 19:38:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:38:08.313625 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c77d14-3b61-49b4-ad8c-c216902bcfdf None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:08.330882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:08.357115 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb5975-4528-4713-91a0-ecf5b404c438 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:08.357436 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb5975-4528-4713-91a0-ecf5b404c438 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:08.357792 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb5975-4528-4713-91a0-ecf5b404c438 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:08.359600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 547/1067] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:38:08.368143 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b01e9-bf4d-4297-aa87-ea2f44b8a87c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:08.368802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b01e9-bf4d-4297-aa87-ea2f44b8a87c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:08.369631 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b01e9-bf4d-4297-aa87-ea2f44b8a87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:08.390713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c75b01e9-bf4d-4297-aa87-ea2f44b8a87c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:08.391978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c75b01e9-bf4d-4297-aa87-ea2f44b8a87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:08.436568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c77d14-3b61-49b4-ad8c-c216902bcfdf None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:08.477379 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b8fad-e270-4b8e-b87f-3608b3293b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:08.478031 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 548/1068] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:38:08.478496 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b8fad-e270-4b8e-b87f-3608b3293b72 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:08.479000 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b8fad-e270-4b8e-b87f-3608b3293b72 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:08.479467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b8fad-e270-4b8e-b87f-3608b3293b72 None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:08.480434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b8fad-e270-4b8e-b87f-3608b3293b72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:08.480911 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b8fad-e270-4b8e-b87f-3608b3293b72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:08.482509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b8fad-e270-4b8e-b87f-3608b3293b72 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:08.503804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 521/1069] 10.222.0.86 () {56 vars in 1133 bytes} [Fri Aug 5 19:38:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:08.504009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b01e9-bf4d-4297-aa87-ea2f44b8a87c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:08.509357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b8fad-e270-4b8e-b87f-3608b3293b72 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:08.570339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:08.594264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:08.627597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b8fad-e270-4b8e-b87f-3608b3293b72 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:08.628729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b01e9-bf4d-4297-aa87-ea2f44b8a87c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:08.664521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2e844-b077-400c-8fad-6789bb8cb5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:08.665165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2e844-b077-400c-8fad-6789bb8cb5ec None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:08.665507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2e844-b077-400c-8fad-6789bb8cb5ec None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:08.665711 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2e844-b077-400c-8fad-6789bb8cb5ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:08.667523 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38aec16-fdda-4a9c-bbd0-82b174dd41c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:08.668167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aec16-fdda-4a9c-bbd0-82b174dd41c9 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:08.668452 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aec16-fdda-4a9c-bbd0-82b174dd41c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:08.668729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aec16-fdda-4a9c-bbd0-82b174dd41c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:08.677168 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2e844-b077-400c-8fad-6789bb8cb5ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:08.678866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2e844-b077-400c-8fad-6789bb8cb5ec None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:08.679937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38aec16-fdda-4a9c-bbd0-82b174dd41c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-a38aec16-fdda-4a9c-bbd0-82b174dd41c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:38:08.684791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:08.685734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:08.686594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:08.686594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:08.686594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:08.686594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:08.686594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:38:08.686594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:38:08.686594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:38:08.686594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:38:08.686594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:38:08.686594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:38:08.686594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:08.686594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:08.686594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 549/1070] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:38:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:08.692325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38aec16-fdda-4a9c-bbd0-82b174dd41c9 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-26b2e844-b077-400c-8fad-6789bb8cb5ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:38:08.697894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:08.698828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:08.699765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:38:08.699765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:38:08.699765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:08.699765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:08.703945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 522/1071] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:38:08.704642 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2e844-b077-400c-8fad-6789bb8cb5ec None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:08.815583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2e844-b077-400c-8fad-6789bb8cb5ec None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:08.817296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38aec16-fdda-4a9c-bbd0-82b174dd41c9 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:08.830066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:08.840138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:08.853552 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1729c2-117b-4640-a02b-a695e0e574c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:08.854167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1729c2-117b-4640-a02b-a695e0e574c9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:08.854440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1729c2-117b-4640-a02b-a695e0e574c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:08.854732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1729c2-117b-4640-a02b-a695e0e574c9 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:08.855413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1729c2-117b-4640-a02b-a695e0e574c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:08.855709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1729c2-117b-4640-a02b-a695e0e574c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:08.857062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1729c2-117b-4640-a02b-a695e0e574c9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:08.860226 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66f60d-9e99-416f-886b-112450da5a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:08.861139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66f60d-9e99-416f-886b-112450da5a76 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:08.861611 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66f60d-9e99-416f-886b-112450da5a76 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:08.862075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66f60d-9e99-416f-886b-112450da5a76 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:08.863003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66f60d-9e99-416f-886b-112450da5a76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:08.863507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66f60d-9e99-416f-886b-112450da5a76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:08.865909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66f60d-9e99-416f-886b-112450da5a76 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:08.869899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 523/1072] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:08.875399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1729c2-117b-4640-a02b-a695e0e574c9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:08.884086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 550/1073] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:38:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:08.885748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66f60d-9e99-416f-886b-112450da5a76 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:08.990612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1729c2-117b-4640-a02b-a695e0e574c9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:08.997185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66f60d-9e99-416f-886b-112450da5a76 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:09.027924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deefa51-59f3-40ba-9653-245e170bbac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:09.028480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deefa51-59f3-40ba-9653-245e170bbac6 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:09.028758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deefa51-59f3-40ba-9653-245e170bbac6 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:09.029027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deefa51-59f3-40ba-9653-245e170bbac6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:09.035048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502deee6-0677-4796-8430-970703e43612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:09.035712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502deee6-0677-4796-8430-970703e43612 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:09.036003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502deee6-0677-4796-8430-970703e43612 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:09.036270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502deee6-0677-4796-8430-970703e43612 None admin] PATH_INFO: `/v3/users/f3879f9dd02248e88c88e8c7e5238370` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:09.036433 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deefa51-59f3-40ba-9653-245e170bbac6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:09.037437 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502deee6-0677-4796-8430-970703e43612 None admin] RBAC: Authorizing `identity:get_user(user_id=f3879f9dd02248e88c88e8c7e5238370)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:09.037756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deefa51-59f3-40ba-9653-245e170bbac6 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:09.039167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502deee6-0677-4796-8430-970703e43612 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:09.045805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 551/1074] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:08 2022] GET /identity/v3/users/f3879f9dd02248e88c88e8c7e5238370 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:09.046450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502deee6-0677-4796-8430-970703e43612 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-9deefa51-59f3-40ba-9653-245e170bbac6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:38:09.052226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:09.053390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:09.054297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:38:09.054297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:38:09.054297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:38:09.054297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:38:09.054297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:38:09.054297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:38:09.054297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:38:09.054297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:09.058764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 524/1075] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:38:09.059119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deefa51-59f3-40ba-9653-245e170bbac6 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:09.092803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:09.132914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:09.165004 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502deee6-0677-4796-8430-970703e43612 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:09.202863 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44b8c2-d889-4676-ab61-e5bec8de48ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:09.203821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44b8c2-d889-4676-ab61-e5bec8de48ec None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:09.204167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deefa51-59f3-40ba-9653-245e170bbac6 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:09.204500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44b8c2-d889-4676-ab61-e5bec8de48ec None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:09.204963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44b8c2-d889-4676-ab61-e5bec8de48ec None admin] PATH_INFO: `/v3/projects/e1b4cd6da17d4492b9220460c4eed905` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:09.206317 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44b8c2-d889-4676-ab61-e5bec8de48ec None admin] RBAC: Authorizing `identity:get_project(project_id=e1b4cd6da17d4492b9220460c4eed905)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:09.208001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44b8c2-d889-4676-ab61-e5bec8de48ec None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:09.214773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 552/1076] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:38:09 2022] GET /identity/v3/projects/e1b4cd6da17d4492b9220460c4eed905 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:09.216024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44b8c2-d889-4676-ab61-e5bec8de48ec None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:09.241987 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334a2c8-fa05-44e4-9f56-30a93742eea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:09.242761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334a2c8-fa05-44e4-9f56-30a93742eea7 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:09.243037 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334a2c8-fa05-44e4-9f56-30a93742eea7 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:09.243375 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334a2c8-fa05-44e4-9f56-30a93742eea7 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:09.244062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334a2c8-fa05-44e4-9f56-30a93742eea7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:09.244363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334a2c8-fa05-44e4-9f56-30a93742eea7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:09.246129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334a2c8-fa05-44e4-9f56-30a93742eea7 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:09.265136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 525/1077] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:38:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:09.266017 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334a2c8-fa05-44e4-9f56-30a93742eea7 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:09.325842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44b8c2-d889-4676-ab61-e5bec8de48ec None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:09.336645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:09.366094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fce4a-494d-4c91-b3ff-5b6d465b8626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:09.366895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fce4a-494d-4c91-b3ff-5b6d465b8626 None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:09.367398 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fce4a-494d-4c91-b3ff-5b6d465b8626 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:09.367526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fce4a-494d-4c91-b3ff-5b6d465b8626 None admin] PATH_INFO: `/v3/projects/e1b4cd6da17d4492b9220460c4eed905/users/f3879f9dd02248e88c88e8c7e5238370/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:09.369405 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fce4a-494d-4c91-b3ff-5b6d465b8626 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1b4cd6da17d4492b9220460c4eed905, user_id=f3879f9dd02248e88c88e8c7e5238370, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:09.370917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fce4a-494d-4c91-b3ff-5b6d465b8626 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:09.385693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334a2c8-fa05-44e4-9f56-30a93742eea7 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:09.387950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:09.389037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 553/1078] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:38:09 2022] PUT /identity/v3/projects/e1b4cd6da17d4492b9220460c4eed905/users/f3879f9dd02248e88c88e8c7e5238370/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:38:09.434215 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cc1c3-97ad-47cc-bcf7-54dde2e4d224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:09.434835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc1c3-97ad-47cc-bcf7-54dde2e4d224 None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:09.435108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc1c3-97ad-47cc-bcf7-54dde2e4d224 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:09.435402 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc1c3-97ad-47cc-bcf7-54dde2e4d224 None admin] PATH_INFO: `/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/ab8f9d9cf4ca4e38ae6eb67e949b40fd/roles/6b65af6de416463283285acc9f0f40f9` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:09.438016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cc1c3-97ad-47cc-bcf7-54dde2e4d224 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c73d2cfaa1c47829ca6936186d2bb26, user_id=ab8f9d9cf4ca4e38ae6eb67e949b40fd, role_id=6b65af6de416463283285acc9f0f40f9)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:09.439543 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cc1c3-97ad-47cc-bcf7-54dde2e4d224 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:09.455604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 526/1079] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:38:09 2022] PUT /identity/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/ab8f9d9cf4ca4e38ae6eb67e949b40fd/roles/6b65af6de416463283285acc9f0f40f9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:38:10.409771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564acd3-7939-4ca5-a2b7-e889123c7389 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:10.410328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564acd3-7939-4ca5-a2b7-e889123c7389 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:10.410328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564acd3-7939-4ca5-a2b7-e889123c7389 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:10.412424 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 554/1080] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:10.417863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63bfc4-2e34-412e-b209-c8d5745561b6 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:10.418281 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63bfc4-2e34-412e-b209-c8d5745561b6 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:10.418483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63bfc4-2e34-412e-b209-c8d5745561b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:10.437553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-2a63bfc4-2e34-412e-b209-c8d5745561b6 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:10.439035 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2a63bfc4-2e34-412e-b209-c8d5745561b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:10.516290 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 527/1081] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:10.539629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564acd3-7939-4ca5-a2b7-e889123c7389 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:10.658832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564acd3-7939-4ca5-a2b7-e889123c7389 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:10.696165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82227057-8b80-462c-81b5-75b8d1e2a082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:10.696872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82227057-8b80-462c-81b5-75b8d1e2a082 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:10.696996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82227057-8b80-462c-81b5-75b8d1e2a082 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:10.697223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82227057-8b80-462c-81b5-75b8d1e2a082 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:10.733383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:10.772518 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82227057-8b80-462c-81b5-75b8d1e2a082 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:10.774142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82227057-8b80-462c-81b5-75b8d1e2a082 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:10.879860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 555/1082] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:38:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:38:10.881581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63bfc4-2e34-412e-b209-c8d5745561b6 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:10.981828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:10.993273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63bfc4-2e34-412e-b209-c8d5745561b6 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:11.028162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bad9a0-18fc-4280-b031-eb52c3391357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:11.028793 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bad9a0-18fc-4280-b031-eb52c3391357 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:11.029067 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bad9a0-18fc-4280-b031-eb52c3391357 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:11.029333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bad9a0-18fc-4280-b031-eb52c3391357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:11.039865 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bad9a0-18fc-4280-b031-eb52c3391357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:11.041296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bad9a0-18fc-4280-b031-eb52c3391357 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-52bad9a0-18fc-4280-b031-eb52c3391357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:38:11.055873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:11.056884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:11.057791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:38:11.057791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:38:11.057791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:11.057791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:11.062045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 528/1083] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:11.063677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82227057-8b80-462c-81b5-75b8d1e2a082 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:11.138818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:11.174772 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82227057-8b80-462c-81b5-75b8d1e2a082 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:11.209932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984a03e-e5da-4451-9dfd-2fedb7d78612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:11.210543 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984a03e-e5da-4451-9dfd-2fedb7d78612 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:11.210844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984a03e-e5da-4451-9dfd-2fedb7d78612 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:11.211123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984a03e-e5da-4451-9dfd-2fedb7d78612 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:11.211809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984a03e-e5da-4451-9dfd-2fedb7d78612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:11.212112 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984a03e-e5da-4451-9dfd-2fedb7d78612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:11.213551 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984a03e-e5da-4451-9dfd-2fedb7d78612 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:11.226612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 556/1084] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:11.487305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135e7a9-3221-4639-89c9-d176dadbee98 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:11.487783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135e7a9-3221-4639-89c9-d176dadbee98 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:11.487861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135e7a9-3221-4639-89c9-d176dadbee98 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:11.489727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 529/1085] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:11.495747 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d4376-c179-4443-80e1-991b4c473b4a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:11.496238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d4376-c179-4443-80e1-991b4c473b4a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:11.496326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d4376-c179-4443-80e1-991b4c473b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:11.524317 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-338d4376-c179-4443-80e1-991b4c473b4a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:11.525562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-338d4376-c179-4443-80e1-991b4c473b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:11.526161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29455e-01c8-4015-83db-ec9187af6f2c None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:11.526537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29455e-01c8-4015-83db-ec9187af6f2c None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:11.527006 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29455e-01c8-4015-83db-ec9187af6f2c None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:11.528545 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 530/1086] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:11.533900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc34126-5ee1-4a2e-beed-03d78eab0b07 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:11.534173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc34126-5ee1-4a2e-beed-03d78eab0b07 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:11.534439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc34126-5ee1-4a2e-beed-03d78eab0b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:11.553425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-3cc34126-5ee1-4a2e-beed-03d78eab0b07 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:11.554630 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3cc34126-5ee1-4a2e-beed-03d78eab0b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:11.599528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 557/1087] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:11.624956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d4376-c179-4443-80e1-991b4c473b4a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:11.638289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 531/1088] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:38:11.662503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc34126-5ee1-4a2e-beed-03d78eab0b07 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:11.684090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:11.728652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d4376-c179-4443-80e1-991b4c473b4a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:11.742317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:11.765610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05608a79-5083-4ace-a2d9-0c869f068aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:11.766247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05608a79-5083-4ace-a2d9-0c869f068aef None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:11.766517 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05608a79-5083-4ace-a2d9-0c869f068aef None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:11.766824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05608a79-5083-4ace-a2d9-0c869f068aef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:11.769948 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc34126-5ee1-4a2e-beed-03d78eab0b07 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:11.777204 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05608a79-5083-4ace-a2d9-0c869f068aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-05608a79-5083-4ace-a2d9-0c869f068aef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:38:11.782034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:11.783359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:11.784933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:11.784933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:11.784933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:11.784933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:11.784933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:38:11.784933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:38:11.784933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:38:11.784933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:38:11.784933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:38:11.784933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:38:11.784933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:11.784933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:11.784933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 558/1089] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:38:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:38:11.789953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05608a79-5083-4ace-a2d9-0c869f068aef None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:11.808295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1671c7-3c8a-4d40-a43e-63ec988e0ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:11.809155 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1671c7-3c8a-4d40-a43e-63ec988e0ff1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:11.809514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1671c7-3c8a-4d40-a43e-63ec988e0ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:11.809894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1671c7-3c8a-4d40-a43e-63ec988e0ff1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:11.825368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1671c7-3c8a-4d40-a43e-63ec988e0ff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-5b1671c7-3c8a-4d40-a43e-63ec988e0ff1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:38:11.833880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:11.842924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 532/1090] 10.222.0.86 () {56 vars in 1120 bytes} [Fri Aug 5 19:38:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:38:11.843883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1671c7-3c8a-4d40-a43e-63ec988e0ff1 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:11.913779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05608a79-5083-4ace-a2d9-0c869f068aef None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:11.933617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:11.952347 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4c812-dd46-4318-9e87-d68070e5ed01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:11.952953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4c812-dd46-4318-9e87-d68070e5ed01 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:11.953221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4c812-dd46-4318-9e87-d68070e5ed01 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:11.953514 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4c812-dd46-4318-9e87-d68070e5ed01 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:11.954211 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4c812-dd46-4318-9e87-d68070e5ed01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:11.954511 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4c812-dd46-4318-9e87-d68070e5ed01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:11.955961 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4c812-dd46-4318-9e87-d68070e5ed01 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:11.964966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 559/1091] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:38:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:11.965245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1671c7-3c8a-4d40-a43e-63ec988e0ff1 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:11.971484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4c812-dd46-4318-9e87-d68070e5ed01 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:12.001375 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0c292-297e-4183-a154-a28cad6d15e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:12.001899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0c292-297e-4183-a154-a28cad6d15e4 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:12.002176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0c292-297e-4183-a154-a28cad6d15e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:12.002437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0c292-297e-4183-a154-a28cad6d15e4 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:12.003181 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0c292-297e-4183-a154-a28cad6d15e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:12.003472 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0c292-297e-4183-a154-a28cad6d15e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:12.005099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0c292-297e-4183-a154-a28cad6d15e4 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:12.006575 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:12.014536 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 533/1092] 10.222.0.86 () {56 vars in 1115 bytes} [Fri Aug 5 19:38:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:12.020947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0c292-297e-4183-a154-a28cad6d15e4 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:12.089742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4c812-dd46-4318-9e87-d68070e5ed01 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:12.124332 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79060e-02f7-44d9-9b0e-81135d8e0c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:12.124957 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79060e-02f7-44d9-9b0e-81135d8e0c5d None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:12.125260 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79060e-02f7-44d9-9b0e-81135d8e0c5d None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:12.125427 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0c292-297e-4183-a154-a28cad6d15e4 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:12.125578 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79060e-02f7-44d9-9b0e-81135d8e0c5d None admin] PATH_INFO: `/v3/users/f3879f9dd02248e88c88e8c7e5238370` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:12.126655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79060e-02f7-44d9-9b0e-81135d8e0c5d None admin] RBAC: Authorizing `identity:get_user(user_id=f3879f9dd02248e88c88e8c7e5238370)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:12.128078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79060e-02f7-44d9-9b0e-81135d8e0c5d None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:12.129859 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 560/1093] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:11 2022] GET /identity/v3/users/f3879f9dd02248e88c88e8c7e5238370 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:12.136175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79060e-02f7-44d9-9b0e-81135d8e0c5d None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:12.164107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c3574-0e5c-4721-a12c-953f2aad277d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:12.164891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c3574-0e5c-4721-a12c-953f2aad277d None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:12.165311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c3574-0e5c-4721-a12c-953f2aad277d None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:12.165680 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c3574-0e5c-4721-a12c-953f2aad277d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:12.176630 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c3574-0e5c-4721-a12c-953f2aad277d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:12.178056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c3574-0e5c-4721-a12c-953f2aad277d None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:12.188244 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-ad0c3574-0e5c-4721-a12c-953f2aad277d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:38:12.194831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:12.196535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:12.198521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:38:12.198521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:38:12.198521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:12.198521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:12.198521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 534/1094] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:38:12.203415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c3574-0e5c-4721-a12c-953f2aad277d None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:12.250032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79060e-02f7-44d9-9b0e-81135d8e0c5d None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:12.261219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:12.291051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce1621-c4e7-42be-bd28-36ea35534bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:12.291671 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce1621-c4e7-42be-bd28-36ea35534bf9 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:12.291946 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce1621-c4e7-42be-bd28-36ea35534bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:12.292212 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce1621-c4e7-42be-bd28-36ea35534bf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:12.316584 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c3574-0e5c-4721-a12c-953f2aad277d None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:12.357258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16245757-e4fa-46ed-b408-7aecc448b53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:12.357867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16245757-e4fa-46ed-b408-7aecc448b53e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:12.358145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16245757-e4fa-46ed-b408-7aecc448b53e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:12.358417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16245757-e4fa-46ed-b408-7aecc448b53e None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:12.359149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16245757-e4fa-46ed-b408-7aecc448b53e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:12.359448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16245757-e4fa-46ed-b408-7aecc448b53e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:12.360849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16245757-e4fa-46ed-b408-7aecc448b53e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:12.372112 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce1621-c4e7-42be-bd28-36ea35534bf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:12.373156 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 535/1095] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:12.377910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce1621-c4e7-42be-bd28-36ea35534bf9 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:12.378888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16245757-e4fa-46ed-b408-7aecc448b53e None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:12.430682 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:12.494800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 561/1096] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:38:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:38:12.500815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce1621-c4e7-42be-bd28-36ea35534bf9 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:12.502878 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16245757-e4fa-46ed-b408-7aecc448b53e None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:12.513545 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:12.540864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275425ed-eb2c-4da5-850e-70881df28723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:12.541504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275425ed-eb2c-4da5-850e-70881df28723 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:12.541783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275425ed-eb2c-4da5-850e-70881df28723 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:12.542049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275425ed-eb2c-4da5-850e-70881df28723 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:12.549492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275425ed-eb2c-4da5-850e-70881df28723 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:12.550970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275425ed-eb2c-4da5-850e-70881df28723 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-275425ed-eb2c-4da5-850e-70881df28723 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:38:12.563615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:12.565007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:12.566176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:12.566176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:12.566176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:38:12.566176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:38:12.566176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:38:12.566176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:38:12.566176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:38:12.566176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:38:12.566176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:38:12.566176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:12.566176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 536/1097] 10.222.0.86 () {56 vars in 1093 bytes} [Fri Aug 5 19:38:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:38:12.573133 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275425ed-eb2c-4da5-850e-70881df28723 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:12.637109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce1621-c4e7-42be-bd28-36ea35534bf9 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:12.674608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35390774-0984-4eb2-9f14-8318403b3102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:12.675264 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35390774-0984-4eb2-9f14-8318403b3102 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:12.675540 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35390774-0984-4eb2-9f14-8318403b3102 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:12.675806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35390774-0984-4eb2-9f14-8318403b3102 None admin] PATH_INFO: `/v3/projects/e1b4cd6da17d4492b9220460c4eed905` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:12.678294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35390774-0984-4eb2-9f14-8318403b3102 None admin] RBAC: Authorizing `identity:get_project(project_id=e1b4cd6da17d4492b9220460c4eed905)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:12.679783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35390774-0984-4eb2-9f14-8318403b3102 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:12.681667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 562/1098] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:38:12 2022] GET /identity/v3/projects/e1b4cd6da17d4492b9220460c4eed905 => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:12.687632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35390774-0984-4eb2-9f14-8318403b3102 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:12.708542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:12.712565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275425ed-eb2c-4da5-850e-70881df28723 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:12.748985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefde207-9a06-4c3d-b7e6-6d54820d6952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:12.749597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefde207-9a06-4c3d-b7e6-6d54820d6952 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:12.749871 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefde207-9a06-4c3d-b7e6-6d54820d6952 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:12.750146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefde207-9a06-4c3d-b7e6-6d54820d6952 None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:12.750961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefde207-9a06-4c3d-b7e6-6d54820d6952 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:12.751264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefde207-9a06-4c3d-b7e6-6d54820d6952 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:12.752680 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefde207-9a06-4c3d-b7e6-6d54820d6952 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:12.771745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 537/1099] 10.222.0.86 () {56 vars in 1133 bytes} [Fri Aug 5 19:38:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:12.777537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefde207-9a06-4c3d-b7e6-6d54820d6952 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:12.807395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:12.811231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35390774-0984-4eb2-9f14-8318403b3102 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:12.846029 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524d124-7b04-4a20-b70d-9a6776246b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:12.846666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524d124-7b04-4a20-b70d-9a6776246b0d None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:12.846947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524d124-7b04-4a20-b70d-9a6776246b0d None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:12.847214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524d124-7b04-4a20-b70d-9a6776246b0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:12.848060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524d124-7b04-4a20-b70d-9a6776246b0d None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=e1b4cd6da17d4492b9220460c4eed905, user.id=f3879f9dd02248e88c88e8c7e5238370) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:12.848862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524d124-7b04-4a20-b70d-9a6776246b0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:12.850379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524d124-7b04-4a20-b70d-9a6776246b0d None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:12.856773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 563/1100] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:38:12 2022] GET /identity/v3/role_assignments?user.id=f3879f9dd02248e88c88e8c7e5238370&scope.project.id=e1b4cd6da17d4492b9220460c4eed905&role.id=93eb204498fc49d297bcd1841e00818d => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:12.888467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefde207-9a06-4c3d-b7e6-6d54820d6952 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:12.933230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e4465-b5c1-477d-9f98-037906cd3d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:12.933860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e4465-b5c1-477d-9f98-037906cd3d63 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:12.934136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e4465-b5c1-477d-9f98-037906cd3d63 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:12.934406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e4465-b5c1-477d-9f98-037906cd3d63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:13.013454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e4465-b5c1-477d-9f98-037906cd3d63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:13.014902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e4465-b5c1-477d-9f98-037906cd3d63 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:13.042449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:13.119795 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 538/1101] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:38:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:38:13.125682 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524d124-7b04-4a20-b70d-9a6776246b0d None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:13.208000 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:13.227922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524d124-7b04-4a20-b70d-9a6776246b0d None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:13.262256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64810413-8ec6-46cb-8024-ebc89ad29c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:13.262921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64810413-8ec6-46cb-8024-ebc89ad29c46 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:13.263197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64810413-8ec6-46cb-8024-ebc89ad29c46 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:13.263464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64810413-8ec6-46cb-8024-ebc89ad29c46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:13.274569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64810413-8ec6-46cb-8024-ebc89ad29c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:13.276029 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64810413-8ec6-46cb-8024-ebc89ad29c46 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-64810413-8ec6-46cb-8024-ebc89ad29c46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:38:13.290019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:13.291213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:13.292130 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:38:13.292130 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:38:13.292130 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:38:13.292130 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:13.292130 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 564/1102] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:13.296961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e4465-b5c1-477d-9f98-037906cd3d63 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:13.398167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e4465-b5c1-477d-9f98-037906cd3d63 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:13.423328 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:13.433402 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77eb31-1b30-4f47-a7a3-2aa121331ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:13.434010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77eb31-1b30-4f47-a7a3-2aa121331ea3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:13.434288 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77eb31-1b30-4f47-a7a3-2aa121331ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:13.434559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77eb31-1b30-4f47-a7a3-2aa121331ea3 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:13.435275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77eb31-1b30-4f47-a7a3-2aa121331ea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:13.435579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77eb31-1b30-4f47-a7a3-2aa121331ea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:13.436944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77eb31-1b30-4f47-a7a3-2aa121331ea3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:13.449090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 539/1103] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:13.454712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64810413-8ec6-46cb-8024-ebc89ad29c46 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:13.557231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64810413-8ec6-46cb-8024-ebc89ad29c46 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:13.588941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:13.595111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0745b84-8c47-4180-84ba-683d5582e4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:13.595810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0745b84-8c47-4180-84ba-683d5582e4d8 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:13.596145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0745b84-8c47-4180-84ba-683d5582e4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:13.596473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0745b84-8c47-4180-84ba-683d5582e4d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:13.605325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0745b84-8c47-4180-84ba-683d5582e4d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:13.606801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0745b84-8c47-4180-84ba-683d5582e4d8 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-e0745b84-8c47-4180-84ba-683d5582e4d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:38:13.619132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:13.620284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:13.621197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:38:13.621197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:38:13.621197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:38:13.621197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:38:13.621197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:38:13.621197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:38:13.621197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:38:13.621197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:13.621197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 565/1104] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:13.625808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77eb31-1b30-4f47-a7a3-2aa121331ea3 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:13.724971 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77eb31-1b30-4f47-a7a3-2aa121331ea3 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:13.757129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aed5e4-429e-491c-9801-eb081d3f67be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:13.757726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aed5e4-429e-491c-9801-eb081d3f67be None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:13.757999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aed5e4-429e-491c-9801-eb081d3f67be None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:13.758348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aed5e4-429e-491c-9801-eb081d3f67be None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:13.759024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aed5e4-429e-491c-9801-eb081d3f67be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:13.759324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aed5e4-429e-491c-9801-eb081d3f67be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:13.760732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aed5e4-429e-491c-9801-eb081d3f67be None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:13.773529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 540/1105] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:38:13.779628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0745b84-8c47-4180-84ba-683d5582e4d8 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:13.878400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0745b84-8c47-4180-84ba-683d5582e4d8 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:13.912401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8415c1c-d337-479b-988f-fc26b76c402c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:13.913201 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8415c1c-d337-479b-988f-fc26b76c402c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:13.913574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8415c1c-d337-479b-988f-fc26b76c402c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:13.913840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8415c1c-d337-479b-988f-fc26b76c402c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:13.914606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8415c1c-d337-479b-988f-fc26b76c402c None admin] RBAC: Adding query filter params (role.id=6b65af6de416463283285acc9f0f40f9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=ab8f9d9cf4ca4e38ae6eb67e949b40fd) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:13.914935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8415c1c-d337-479b-988f-fc26b76c402c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:13.916348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8415c1c-d337-479b-988f-fc26b76c402c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:13.930656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 566/1106] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:38:13 2022] GET /identity/v3/role_assignments?user.id=ab8f9d9cf4ca4e38ae6eb67e949b40fd&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=6b65af6de416463283285acc9f0f40f9 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:14.931044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea24b6d-add6-4979-af6b-8812c9912906 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:14.931579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea24b6d-add6-4979-af6b-8812c9912906 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:14.931579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea24b6d-add6-4979-af6b-8812c9912906 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:14.932897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 541/1107] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:38:14.938227 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efa652-5b0c-4abf-8d30-395db4c4e709 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:14.938666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efa652-5b0c-4abf-8d30-395db4c4e709 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:14.938797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efa652-5b0c-4abf-8d30-395db4c4e709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:14.957731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-f7efa652-5b0c-4abf-8d30-395db4c4e709 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:14.959085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f7efa652-5b0c-4abf-8d30-395db4c4e709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:15.011066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:15.027850 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 567/1108] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:15.053344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea24b6d-add6-4979-af6b-8812c9912906 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:15.101629 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:15.154775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea24b6d-add6-4979-af6b-8812c9912906 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:15.189026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a0621-523b-4bf9-a95a-d1bbf58314e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:15.189586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a0621-523b-4bf9-a95a-d1bbf58314e9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:15.189863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a0621-523b-4bf9-a95a-d1bbf58314e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:15.190133 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a0621-523b-4bf9-a95a-d1bbf58314e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:15.191299 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a0621-523b-4bf9-a95a-d1bbf58314e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:15.192826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a0621-523b-4bf9-a95a-d1bbf58314e9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:15.194411 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 542/1109] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:15.200329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7efa652-5b0c-4abf-8d30-395db4c4e709 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:15.300022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7efa652-5b0c-4abf-8d30-395db4c4e709 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:15.336226 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4222e3b-92d2-4897-904f-b73a234ab61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:15.336836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4222e3b-92d2-4897-904f-b73a234ab61f None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:15.337113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4222e3b-92d2-4897-904f-b73a234ab61f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:15.337385 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4222e3b-92d2-4897-904f-b73a234ab61f None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:15.338057 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4222e3b-92d2-4897-904f-b73a234ab61f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:15.339594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4222e3b-92d2-4897-904f-b73a234ab61f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:15.359993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 568/1110] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:38:15 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:38:17.400682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f10265-fdc0-4ad1-aa2b-e72f0e395f69 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:17.401262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f10265-fdc0-4ad1-aa2b-e72f0e395f69 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:17.401262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f10265-fdc0-4ad1-aa2b-e72f0e395f69 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:17.402532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 543/1111] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:17.407912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9c30e-5906-4540-ab42-fe951e69d0e0 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:17.408359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9c30e-5906-4540-ab42-fe951e69d0e0 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:17.408447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9c30e-5906-4540-ab42-fe951e69d0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:17.427423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-0ad9c30e-5906-4540-ab42-fe951e69d0e0 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:17.428779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-0ad9c30e-5906-4540-ab42-fe951e69d0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:17.463383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:17.494949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 569/1112] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:17.518736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f10265-fdc0-4ad1-aa2b-e72f0e395f69 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:17.623517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f10265-fdc0-4ad1-aa2b-e72f0e395f69 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:17.637027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:17.656905 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baba8a6-a63e-4e47-9078-41432cfe8989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:17.657481 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baba8a6-a63e-4e47-9078-41432cfe8989 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:17.657754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baba8a6-a63e-4e47-9078-41432cfe8989 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:17.658017 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baba8a6-a63e-4e47-9078-41432cfe8989 None admin] PATH_INFO: `/v3/roles/686764f0686e446f884315081e158bf3` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:17.659451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baba8a6-a63e-4e47-9078-41432cfe8989 None admin] RBAC: Authorizing `identity:get_role(role_id=686764f0686e446f884315081e158bf3)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:17.661340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baba8a6-a63e-4e47-9078-41432cfe8989 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:17.662645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 544/1113] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:17 2022] GET /identity/v3/roles/686764f0686e446f884315081e158bf3 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:17.668538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9c30e-5906-4540-ab42-fe951e69d0e0 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:17.765371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9c30e-5906-4540-ab42-fe951e69d0e0 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:17.798499 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf2f80-bc10-448f-a357-ab9077853cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:17.799157 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf2f80-bc10-448f-a357-ab9077853cc9 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:17.799430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf2f80-bc10-448f-a357-ab9077853cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:17.799699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf2f80-bc10-448f-a357-ab9077853cc9 None admin] PATH_INFO: `/v3/users/0b5276bc5852411aa571df028d4635c3` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:17.813651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf2f80-bc10-448f-a357-ab9077853cc9 None admin] RBAC: Authorizing `identity:get_user(user_id=0b5276bc5852411aa571df028d4635c3)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:17.815119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf2f80-bc10-448f-a357-ab9077853cc9 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:17.816819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 570/1114] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:17 2022] GET /identity/v3/users/0b5276bc5852411aa571df028d4635c3 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:17.822889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baba8a6-a63e-4e47-9078-41432cfe8989 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:17.927208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baba8a6-a63e-4e47-9078-41432cfe8989 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:17.961173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3494d65-ae70-40e4-b238-b2d69ea01c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:17.961775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3494d65-ae70-40e4-b238-b2d69ea01c64 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:17.962051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3494d65-ae70-40e4-b238-b2d69ea01c64 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:17.962319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3494d65-ae70-40e4-b238-b2d69ea01c64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:18.009440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:18.027216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3494d65-ae70-40e4-b238-b2d69ea01c64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:18.028643 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3494d65-ae70-40e4-b238-b2d69ea01c64 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:18.127481 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 545/1115] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:38:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:38:18.134785 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf2f80-bc10-448f-a357-ab9077853cc9 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:18.164715 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:18.234019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf2f80-bc10-448f-a357-ab9077853cc9 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:18.267778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7199e57-2702-433e-b825-36fc91ef7cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:18.268396 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7199e57-2702-433e-b825-36fc91ef7cfa None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:18.268669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7199e57-2702-433e-b825-36fc91ef7cfa None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:18.268939 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7199e57-2702-433e-b825-36fc91ef7cfa None admin] PATH_INFO: `/v3/projects/e1b4cd6da17d4492b9220460c4eed905` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:18.270509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7199e57-2702-433e-b825-36fc91ef7cfa None admin] RBAC: Authorizing `identity:get_project(project_id=e1b4cd6da17d4492b9220460c4eed905)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:18.271945 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7199e57-2702-433e-b825-36fc91ef7cfa None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:18.273605 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 571/1116] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:38:18 2022] GET /identity/v3/projects/e1b4cd6da17d4492b9220460c4eed905 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:18.279885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3494d65-ae70-40e4-b238-b2d69ea01c64 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:18.359781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:18.378317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3494d65-ae70-40e4-b238-b2d69ea01c64 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:18.414416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee75a8-8f92-4e90-97fc-b591d7ff8a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:18.415150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee75a8-8f92-4e90-97fc-b591d7ff8a66 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:18.415318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee75a8-8f92-4e90-97fc-b591d7ff8a66 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:18.415587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee75a8-8f92-4e90-97fc-b591d7ff8a66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:18.416362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee75a8-8f92-4e90-97fc-b591d7ff8a66 None admin] RBAC: Adding query filter params (role.id=686764f0686e446f884315081e158bf3, scope.project.id=e1b4cd6da17d4492b9220460c4eed905, user.id=0b5276bc5852411aa571df028d4635c3) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:18.416668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee75a8-8f92-4e90-97fc-b591d7ff8a66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:18.418089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee75a8-8f92-4e90-97fc-b591d7ff8a66 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:18.423484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 546/1117] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:38:18 2022] GET /identity/v3/role_assignments?user.id=0b5276bc5852411aa571df028d4635c3&scope.project.id=e1b4cd6da17d4492b9220460c4eed905&role.id=686764f0686e446f884315081e158bf3 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:20.407980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad9563-b6ce-4f8c-8043-93cffd767afb None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:20.408515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad9563-b6ce-4f8c-8043-93cffd767afb None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:20.408515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad9563-b6ce-4f8c-8043-93cffd767afb None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:20.409878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 572/1118] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:20.415138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4b4d9-4201-4d0b-885b-d399c1527f48 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:20.415563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4b4d9-4201-4d0b-885b-d399c1527f48 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:20.415672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4b4d9-4201-4d0b-885b-d399c1527f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:20.435061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-23d4b4d9-4201-4d0b-885b-d399c1527f48 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:20.436251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-23d4b4d9-4201-4d0b-885b-d399c1527f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:20.507383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 547/1119] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:20.530686 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad9563-b6ce-4f8c-8043-93cffd767afb None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:20.627719 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:20.633151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad9563-b6ce-4f8c-8043-93cffd767afb None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:20.667109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f85d2-b931-4468-8437-1740b048880a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:20.667656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f85d2-b931-4468-8437-1740b048880a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:20.667927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f85d2-b931-4468-8437-1740b048880a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:20.668192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f85d2-b931-4468-8437-1740b048880a None admin] PATH_INFO: `/v3/roles/686764f0686e446f884315081e158bf3` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:20.669620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f85d2-b931-4468-8437-1740b048880a None admin] RBAC: Authorizing `identity:get_role(role_id=686764f0686e446f884315081e158bf3)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:20.671014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f85d2-b931-4468-8437-1740b048880a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:20.672365 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 573/1120] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:20 2022] GET /identity/v3/roles/686764f0686e446f884315081e158bf3 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:20.679141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4b4d9-4201-4d0b-885b-d399c1527f48 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:20.755352 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:20.777563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4b4d9-4201-4d0b-885b-d399c1527f48 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:20.811325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3110d-54b1-4e3b-b64e-10a72e152237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:20.811948 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3110d-54b1-4e3b-b64e-10a72e152237 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:20.812223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3110d-54b1-4e3b-b64e-10a72e152237 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:20.812489 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3110d-54b1-4e3b-b64e-10a72e152237 None admin] PATH_INFO: `/v3/users/0b5276bc5852411aa571df028d4635c3` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:20.813888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3110d-54b1-4e3b-b64e-10a72e152237 None admin] RBAC: Authorizing `identity:get_user(user_id=0b5276bc5852411aa571df028d4635c3)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:20.815392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3110d-54b1-4e3b-b64e-10a72e152237 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:20.817031 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 548/1121] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:20 2022] GET /identity/v3/users/0b5276bc5852411aa571df028d4635c3 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:20.822971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f85d2-b931-4468-8437-1740b048880a None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:20.922558 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f85d2-b931-4468-8437-1740b048880a None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:20.956623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478cb36-960f-44a4-97c5-62dc606a0b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:20.957420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478cb36-960f-44a4-97c5-62dc606a0b15 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:20.957844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478cb36-960f-44a4-97c5-62dc606a0b15 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:20.958114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478cb36-960f-44a4-97c5-62dc606a0b15 None admin] PATH_INFO: `/v3/projects/e1b4cd6da17d4492b9220460c4eed905` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:20.959262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478cb36-960f-44a4-97c5-62dc606a0b15 None admin] RBAC: Authorizing `identity:get_project(project_id=e1b4cd6da17d4492b9220460c4eed905)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:20.960708 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478cb36-960f-44a4-97c5-62dc606a0b15 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:20.962366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 574/1122] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:38:20 2022] GET /identity/v3/projects/e1b4cd6da17d4492b9220460c4eed905 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:20.968327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3110d-54b1-4e3b-b64e-10a72e152237 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:21.076552 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3110d-54b1-4e3b-b64e-10a72e152237 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:21.109411 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb0969-89be-4ad2-9c46-2cb3d8594a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:21.110061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb0969-89be-4ad2-9c46-2cb3d8594a0a None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:21.110338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb0969-89be-4ad2-9c46-2cb3d8594a0a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:21.110609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb0969-89be-4ad2-9c46-2cb3d8594a0a None admin] PATH_INFO: `/v3/projects/e1b4cd6da17d4492b9220460c4eed905/users/0b5276bc5852411aa571df028d4635c3/roles/686764f0686e446f884315081e158bf3` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:21.112794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb0969-89be-4ad2-9c46-2cb3d8594a0a None admin] RBAC: Authorizing `identity:create_grant(project_id=e1b4cd6da17d4492b9220460c4eed905, user_id=0b5276bc5852411aa571df028d4635c3, role_id=686764f0686e446f884315081e158bf3)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:21.115049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb0969-89be-4ad2-9c46-2cb3d8594a0a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:21.127840 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 549/1123] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:38:20 2022] PUT /identity/v3/projects/e1b4cd6da17d4492b9220460c4eed905/users/0b5276bc5852411aa571df028d4635c3/roles/686764f0686e446f884315081e158bf3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:38:23.135881 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04837e65-a005-4e5b-9d09-3ec52a80f3ce None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:23.136419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04837e65-a005-4e5b-9d09-3ec52a80f3ce None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:23.136419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04837e65-a005-4e5b-9d09-3ec52a80f3ce None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:23.137758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 575/1124] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:23.143277 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15359ab5-f672-4bd1-b354-842ed86fff36 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:23.143716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15359ab5-f672-4bd1-b354-842ed86fff36 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:23.143828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15359ab5-f672-4bd1-b354-842ed86fff36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:23.151972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:23.164256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-15359ab5-f672-4bd1-b354-842ed86fff36 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:23.165466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-15359ab5-f672-4bd1-b354-842ed86fff36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:23.235093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 550/1125] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:23.259710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04837e65-a005-4e5b-9d09-3ec52a80f3ce None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:23.296939 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:23.369277 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04837e65-a005-4e5b-9d09-3ec52a80f3ce None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:23.403252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574350d6-f556-4ec0-aad6-ef24e730a19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:23.403767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574350d6-f556-4ec0-aad6-ef24e730a19f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:23.404038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574350d6-f556-4ec0-aad6-ef24e730a19f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:23.404300 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574350d6-f556-4ec0-aad6-ef24e730a19f None admin] PATH_INFO: `/v3/roles/686764f0686e446f884315081e158bf3` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:23.405730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574350d6-f556-4ec0-aad6-ef24e730a19f None admin] RBAC: Authorizing `identity:get_role(role_id=686764f0686e446f884315081e158bf3)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:23.407261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574350d6-f556-4ec0-aad6-ef24e730a19f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:23.408623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 576/1126] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:23 2022] GET /identity/v3/roles/686764f0686e446f884315081e158bf3 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:23.415493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15359ab5-f672-4bd1-b354-842ed86fff36 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:23.517535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15359ab5-f672-4bd1-b354-842ed86fff36 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:23.553848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75cbb2-3919-4de8-bbf1-a46bf03b75a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:23.554474 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75cbb2-3919-4de8-bbf1-a46bf03b75a1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:23.554851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75cbb2-3919-4de8-bbf1-a46bf03b75a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:23.555053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75cbb2-3919-4de8-bbf1-a46bf03b75a1 None admin] PATH_INFO: `/v3/users/0b5276bc5852411aa571df028d4635c3` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:23.556202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75cbb2-3919-4de8-bbf1-a46bf03b75a1 None admin] RBAC: Authorizing `identity:get_user(user_id=0b5276bc5852411aa571df028d4635c3)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:23.557624 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75cbb2-3919-4de8-bbf1-a46bf03b75a1 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:23.559254 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 551/1127] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:23 2022] GET /identity/v3/users/0b5276bc5852411aa571df028d4635c3 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:23.565399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574350d6-f556-4ec0-aad6-ef24e730a19f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:23.669572 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574350d6-f556-4ec0-aad6-ef24e730a19f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:23.671050 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:23.705483 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b4067-01c3-4a33-8c85-d13e0a50dfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:23.706087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b4067-01c3-4a33-8c85-d13e0a50dfc7 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:23.706360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b4067-01c3-4a33-8c85-d13e0a50dfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:23.706649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b4067-01c3-4a33-8c85-d13e0a50dfc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:23.776304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b4067-01c3-4a33-8c85-d13e0a50dfc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:23.777741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b4067-01c3-4a33-8c85-d13e0a50dfc7 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:23.878406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 577/1128] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:38:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:38:23.884596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75cbb2-3919-4de8-bbf1-a46bf03b75a1 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:23.887525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:23.986029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75cbb2-3919-4de8-bbf1-a46bf03b75a1 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:24.022859 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7f256-a70c-4337-8daf-3ad461b23c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:24.023506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7f256-a70c-4337-8daf-3ad461b23c2b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:24.023781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7f256-a70c-4337-8daf-3ad461b23c2b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:24.024053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7f256-a70c-4337-8daf-3ad461b23c2b None admin] PATH_INFO: `/v3/projects/e1b4cd6da17d4492b9220460c4eed905` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:24.025253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7f256-a70c-4337-8daf-3ad461b23c2b None admin] RBAC: Authorizing `identity:get_project(project_id=e1b4cd6da17d4492b9220460c4eed905)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:24.026708 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7f256-a70c-4337-8daf-3ad461b23c2b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:24.028391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 552/1129] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:38:23 2022] GET /identity/v3/projects/e1b4cd6da17d4492b9220460c4eed905 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:24.034673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b4067-01c3-4a33-8c85-d13e0a50dfc7 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:24.036942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:24.143880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b4067-01c3-4a33-8c85-d13e0a50dfc7 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:24.178540 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76492907-4feb-416a-9477-630cad1062e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:24.179182 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76492907-4feb-416a-9477-630cad1062e6 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:24.179461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76492907-4feb-416a-9477-630cad1062e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:24.179734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76492907-4feb-416a-9477-630cad1062e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:24.180526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76492907-4feb-416a-9477-630cad1062e6 None admin] RBAC: Adding query filter params (role.id=686764f0686e446f884315081e158bf3, scope.project.id=e1b4cd6da17d4492b9220460c4eed905, user.id=0b5276bc5852411aa571df028d4635c3) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:24.180837 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76492907-4feb-416a-9477-630cad1062e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:24.182268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76492907-4feb-416a-9477-630cad1062e6 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:24.188532 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 578/1130] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:38:24 2022] GET /identity/v3/role_assignments?user.id=0b5276bc5852411aa571df028d4635c3&scope.project.id=e1b4cd6da17d4492b9220460c4eed905&role.id=686764f0686e446f884315081e158bf3 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:26.268954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6123d2f-db17-4189-995b-80a5e3d9637f None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:26.269517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6123d2f-db17-4189-995b-80a5e3d9637f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:26.269665 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6123d2f-db17-4189-995b-80a5e3d9637f None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:26.271040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 553/1131] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:26.276161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d58511-4e45-467d-839b-7784156b3f2d None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:26.276681 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d58511-4e45-467d-839b-7784156b3f2d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:26.276778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d58511-4e45-467d-839b-7784156b3f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:26.294934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-51d58511-4e45-467d-839b-7784156b3f2d None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:26.296124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-51d58511-4e45-467d-839b-7784156b3f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:26.354813 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:26.362752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 579/1132] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:38:26.386467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6123d2f-db17-4189-995b-80a5e3d9637f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:26.457293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:26.489449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6123d2f-db17-4189-995b-80a5e3d9637f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:26.523509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4567c61-25f6-4a59-93c2-c082e296ee78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:26.524033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4567c61-25f6-4a59-93c2-c082e296ee78 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:26.524313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4567c61-25f6-4a59-93c2-c082e296ee78 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:26.524582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4567c61-25f6-4a59-93c2-c082e296ee78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:26.525722 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4567c61-25f6-4a59-93c2-c082e296ee78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:26.527202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4567c61-25f6-4a59-93c2-c082e296ee78 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:26.528793 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 554/1133] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:26.534764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d58511-4e45-467d-839b-7784156b3f2d None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:26.635572 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d58511-4e45-467d-839b-7784156b3f2d None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:26.669558 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea8275-e01f-43d5-8249-5f0197935e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:26.670167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea8275-e01f-43d5-8249-5f0197935e88 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:26.670439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea8275-e01f-43d5-8249-5f0197935e88 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:26.670732 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea8275-e01f-43d5-8249-5f0197935e88 None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:26.671400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea8275-e01f-43d5-8249-5f0197935e88 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:26.672802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea8275-e01f-43d5-8249-5f0197935e88 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:26.700309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 580/1134] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:38:26 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:38:28.854389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02e20f-7325-46d5-b5af-9ee7549b2082 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:28.854975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02e20f-7325-46d5-b5af-9ee7549b2082 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:28.854975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02e20f-7325-46d5-b5af-9ee7549b2082 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:28.856292 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 555/1135] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:28.861875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63037ae2-75cf-4d1b-bfd9-043a3e46d245 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:28.862311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63037ae2-75cf-4d1b-bfd9-043a3e46d245 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:28.862468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63037ae2-75cf-4d1b-bfd9-043a3e46d245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:28.881072 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-63037ae2-75cf-4d1b-bfd9-043a3e46d245 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:28.882243 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-63037ae2-75cf-4d1b-bfd9-043a3e46d245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:28.924942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:28.950470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 581/1136] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:28.975750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02e20f-7325-46d5-b5af-9ee7549b2082 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:29.076405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02e20f-7325-46d5-b5af-9ee7549b2082 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:29.109774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e59e8f-2208-4d44-a4b6-82aa52ae5255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:29.110325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e59e8f-2208-4d44-a4b6-82aa52ae5255 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:29.110604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e59e8f-2208-4d44-a4b6-82aa52ae5255 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:29.110902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e59e8f-2208-4d44-a4b6-82aa52ae5255 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:29.112044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e59e8f-2208-4d44-a4b6-82aa52ae5255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:29.114409 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e59e8f-2208-4d44-a4b6-82aa52ae5255 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:29.116034 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 556/1137] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:38:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:29.121801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63037ae2-75cf-4d1b-bfd9-043a3e46d245 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:29.219080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63037ae2-75cf-4d1b-bfd9-043a3e46d245 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:29.252351 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95df393-7d84-4b99-a4f3-530cbdb05cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:29.252966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95df393-7d84-4b99-a4f3-530cbdb05cd4 None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:29.253235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95df393-7d84-4b99-a4f3-530cbdb05cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:29.253500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95df393-7d84-4b99-a4f3-530cbdb05cd4 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:29.254183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95df393-7d84-4b99-a4f3-530cbdb05cd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:29.255636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95df393-7d84-4b99-a4f3-530cbdb05cd4 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:29.277095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 582/1138] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:38:29 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:38:31.381251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b92bdb-f479-4f9b-9f13-1835c9988228 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:31.381790 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b92bdb-f479-4f9b-9f13-1835c9988228 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:31.381790 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b92bdb-f479-4f9b-9f13-1835c9988228 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:31.383166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 557/1139] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:31.388412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cc641-ac1d-4548-bfd4-35be5c4d2155 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:31.388817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cc641-ac1d-4548-bfd4-35be5c4d2155 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:31.388973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1cc641-ac1d-4548-bfd4-35be5c4d2155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:31.407822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-fd1cc641-ac1d-4548-bfd4-35be5c4d2155 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:31.409010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fd1cc641-ac1d-4548-bfd4-35be5c4d2155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:31.435518 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:31.475875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 583/1140] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:31.498615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b92bdb-f479-4f9b-9f13-1835c9988228 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:31.507690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:31.599733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b92bdb-f479-4f9b-9f13-1835c9988228 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:31.633439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc05dc-99f7-432c-9c13-4d35267692ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:31.633992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc05dc-99f7-432c-9c13-4d35267692ad None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:31.634267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc05dc-99f7-432c-9c13-4d35267692ad None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:31.634535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc05dc-99f7-432c-9c13-4d35267692ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:31.644984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc05dc-99f7-432c-9c13-4d35267692ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-2fcc05dc-99f7-432c-9c13-4d35267692ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:38:31.649906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:31.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:31.651880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:31.651880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:31.651880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:31.651880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:31.651880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:38:31.651880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:38:31.651880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:38:31.651880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:38:31.651880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:38:31.651880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:38:31.651880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:31.651880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:31.651880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 558/1141] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:38:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:31.657289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cc641-ac1d-4548-bfd4-35be5c4d2155 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:31.752663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1cc641-ac1d-4548-bfd4-35be5c4d2155 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:31.788165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be675e-d6ad-4c39-9933-6d036ac58461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:31.788782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be675e-d6ad-4c39-9933-6d036ac58461 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:31.789055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be675e-d6ad-4c39-9933-6d036ac58461 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:31.789322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be675e-d6ad-4c39-9933-6d036ac58461 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:31.790031 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be675e-d6ad-4c39-9933-6d036ac58461 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:31.790334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be675e-d6ad-4c39-9933-6d036ac58461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:31.791725 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be675e-d6ad-4c39-9933-6d036ac58461 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:31.803601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 584/1142] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:38:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:31.809543 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc05dc-99f7-432c-9c13-4d35267692ad None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:31.873840 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:31.907050 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc05dc-99f7-432c-9c13-4d35267692ad None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:31.940506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e24280-9e70-4613-847c-52be905f6aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:31.941290 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e24280-9e70-4613-847c-52be905f6aaf None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:31.941500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e24280-9e70-4613-847c-52be905f6aaf None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:31.941767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e24280-9e70-4613-847c-52be905f6aaf None admin] PATH_INFO: `/v3/users/4a73aa603af94c53ab054a5fd9a001d8` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:31.954387 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e24280-9e70-4613-847c-52be905f6aaf None admin] RBAC: Authorizing `identity:get_user(user_id=4a73aa603af94c53ab054a5fd9a001d8)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:31.955842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e24280-9e70-4613-847c-52be905f6aaf None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:31.957614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 559/1143] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:31 2022] GET /identity/v3/users/4a73aa603af94c53ab054a5fd9a001d8 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:31.963492 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be675e-d6ad-4c39-9933-6d036ac58461 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:31.992564 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:32.062374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be675e-d6ad-4c39-9933-6d036ac58461 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:32.095445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8849c24-5e0a-4b11-8b58-232b43c9bd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:32.096044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8849c24-5e0a-4b11-8b58-232b43c9bd35 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:32.096318 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8849c24-5e0a-4b11-8b58-232b43c9bd35 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:32.096584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8849c24-5e0a-4b11-8b58-232b43c9bd35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:32.163194 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8849c24-5e0a-4b11-8b58-232b43c9bd35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:32.164551 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8849c24-5e0a-4b11-8b58-232b43c9bd35 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:32.190064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:32.262145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 585/1144] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:38:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:38:32.268054 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e24280-9e70-4613-847c-52be905f6aaf None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:32.366366 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e24280-9e70-4613-847c-52be905f6aaf None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:32.388946 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:32.400617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ced330-a77d-423a-bd88-d2571aaf3b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:32.401244 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ced330-a77d-423a-bd88-d2571aaf3b98 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:32.401520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ced330-a77d-423a-bd88-d2571aaf3b98 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:32.401789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ced330-a77d-423a-bd88-d2571aaf3b98 None admin] PATH_INFO: `/v3/projects/d752974e97c749be9f867eba56b1bbef` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:32.403103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ced330-a77d-423a-bd88-d2571aaf3b98 None admin] RBAC: Authorizing `identity:get_project(project_id=d752974e97c749be9f867eba56b1bbef)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:32.404399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ced330-a77d-423a-bd88-d2571aaf3b98 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:32.406075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 560/1145] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:38:32 2022] GET /identity/v3/projects/d752974e97c749be9f867eba56b1bbef => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:32.413158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8849c24-5e0a-4b11-8b58-232b43c9bd35 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:32.514726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8849c24-5e0a-4b11-8b58-232b43c9bd35 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:32.540677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:32.549489 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213f33e-415b-486b-a352-9b345f88b9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:32.550096 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213f33e-415b-486b-a352-9b345f88b9c8 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:32.550582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213f33e-415b-486b-a352-9b345f88b9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:32.550908 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213f33e-415b-486b-a352-9b345f88b9c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:32.551676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213f33e-415b-486b-a352-9b345f88b9c8 None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=d752974e97c749be9f867eba56b1bbef, user.id=4a73aa603af94c53ab054a5fd9a001d8) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:32.551981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213f33e-415b-486b-a352-9b345f88b9c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:32.553391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213f33e-415b-486b-a352-9b345f88b9c8 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:32.558955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 586/1146] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:38:32 2022] GET /identity/v3/role_assignments?user.id=4a73aa603af94c53ab054a5fd9a001d8&scope.project.id=d752974e97c749be9f867eba56b1bbef&role.id=93eb204498fc49d297bcd1841e00818d => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:34.545845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab3402-4525-449c-96bc-53bce9ceeed6 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:34.546359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab3402-4525-449c-96bc-53bce9ceeed6 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:34.546359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab3402-4525-449c-96bc-53bce9ceeed6 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:34.547753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 561/1147] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:34.553198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2858cf3-77de-42f4-b476-7f5bb65cd88f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:34.553623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2858cf3-77de-42f4-b476-7f5bb65cd88f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:34.553753 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2858cf3-77de-42f4-b476-7f5bb65cd88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:34.571510 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e2858cf3-77de-42f4-b476-7f5bb65cd88f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:34.572685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e2858cf3-77de-42f4-b476-7f5bb65cd88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:34.638020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 587/1148] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:34.662580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab3402-4525-449c-96bc-53bce9ceeed6 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:34.762271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab3402-4525-449c-96bc-53bce9ceeed6 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:34.796088 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a01c5-6820-4047-984e-bdb4288da734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:34.796707 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a01c5-6820-4047-984e-bdb4288da734 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:34.796981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a01c5-6820-4047-984e-bdb4288da734 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:34.797249 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a01c5-6820-4047-984e-bdb4288da734 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:34.807119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a01c5-6820-4047-984e-bdb4288da734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-3e2a01c5-6820-4047-984e-bdb4288da734 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:38:34.811942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:34.813323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:34.814368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:34.814368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:34.814368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:34.814368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:34.814368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:38:34.814368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:38:34.814368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:38:34.814368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:38:34.814368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:38:34.814368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:38:34.814368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:34.814368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:34.814368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 562/1149] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:38:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:34.820173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2858cf3-77de-42f4-b476-7f5bb65cd88f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:34.917557 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2858cf3-77de-42f4-b476-7f5bb65cd88f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:34.936811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:34.951919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1218c3-24c6-4572-95ea-e604e5287374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:34.952527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1218c3-24c6-4572-95ea-e604e5287374 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:34.952806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1218c3-24c6-4572-95ea-e604e5287374 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:34.953074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1218c3-24c6-4572-95ea-e604e5287374 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:34.953771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1218c3-24c6-4572-95ea-e604e5287374 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:34.954070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1218c3-24c6-4572-95ea-e604e5287374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:34.955502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1218c3-24c6-4572-95ea-e604e5287374 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:34.964084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 588/1150] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:38:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:34.970025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a01c5-6820-4047-984e-bdb4288da734 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:35.023791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:35.070786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a01c5-6820-4047-984e-bdb4288da734 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:35.104010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b0602-2a44-4ddf-930c-a3e310587baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:35.104634 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b0602-2a44-4ddf-930c-a3e310587baa None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:35.104912 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b0602-2a44-4ddf-930c-a3e310587baa None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:35.105183 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b0602-2a44-4ddf-930c-a3e310587baa None admin] PATH_INFO: `/v3/users/4a73aa603af94c53ab054a5fd9a001d8` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:35.106334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b0602-2a44-4ddf-930c-a3e310587baa None admin] RBAC: Authorizing `identity:get_user(user_id=4a73aa603af94c53ab054a5fd9a001d8)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:35.107823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b0602-2a44-4ddf-930c-a3e310587baa None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:35.109417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 563/1151] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:34 2022] GET /identity/v3/users/4a73aa603af94c53ab054a5fd9a001d8 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:35.115351 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1218c3-24c6-4572-95ea-e604e5287374 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:35.218054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1218c3-24c6-4572-95ea-e604e5287374 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:35.252058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429eb3d0-0071-463d-8bf3-167eadb1f7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:35.252688 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429eb3d0-0071-463d-8bf3-167eadb1f7f1 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:35.252966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429eb3d0-0071-463d-8bf3-167eadb1f7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:35.253238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429eb3d0-0071-463d-8bf3-167eadb1f7f1 None admin] PATH_INFO: `/v3/projects/d752974e97c749be9f867eba56b1bbef` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:35.255123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429eb3d0-0071-463d-8bf3-167eadb1f7f1 None admin] RBAC: Authorizing `identity:get_project(project_id=d752974e97c749be9f867eba56b1bbef)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:35.256546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429eb3d0-0071-463d-8bf3-167eadb1f7f1 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:35.258231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 589/1152] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:38:35 2022] GET /identity/v3/projects/d752974e97c749be9f867eba56b1bbef => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:35.264237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b0602-2a44-4ddf-930c-a3e310587baa None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:35.359612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b0602-2a44-4ddf-930c-a3e310587baa None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:35.380041 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:35.393693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f00eb-b2a7-4815-b144-4c84b139ca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:35.394374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f00eb-b2a7-4815-b144-4c84b139ca5a None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:35.394763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f00eb-b2a7-4815-b144-4c84b139ca5a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:35.395038 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f00eb-b2a7-4815-b144-4c84b139ca5a None admin] PATH_INFO: `/v3/projects/d752974e97c749be9f867eba56b1bbef/users/4a73aa603af94c53ab054a5fd9a001d8/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:35.396992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f00eb-b2a7-4815-b144-4c84b139ca5a None admin] RBAC: Authorizing `identity:create_grant(project_id=d752974e97c749be9f867eba56b1bbef, user_id=4a73aa603af94c53ab054a5fd9a001d8, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:35.398409 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f00eb-b2a7-4815-b144-4c84b139ca5a None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:35.412083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 564/1153] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:38:35 2022] PUT /identity/v3/projects/d752974e97c749be9f867eba56b1bbef/users/4a73aa603af94c53ab054a5fd9a001d8/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:38:37.485995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a2852-6b3c-44ae-8ce2-d61749f7e6ce None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:37.486527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a2852-6b3c-44ae-8ce2-d61749f7e6ce None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:37.486527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a2852-6b3c-44ae-8ce2-d61749f7e6ce None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:37.488454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 590/1154] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:37.494276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb023d0-43d0-4729-b1bc-1539efc7039f None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:37.494761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb023d0-43d0-4729-b1bc-1539efc7039f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:37.494851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb023d0-43d0-4729-b1bc-1539efc7039f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:37.515476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-9eb023d0-43d0-4729-b1bc-1539efc7039f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:37.516680 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9eb023d0-43d0-4729-b1bc-1539efc7039f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:37.582763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 565/1155] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:37.605700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a2852-6b3c-44ae-8ce2-d61749f7e6ce None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:37.655658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:37.707523 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a2852-6b3c-44ae-8ce2-d61749f7e6ce None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:37.740640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2395e1-ddf5-4bd5-8730-ef76176cf426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:37.741196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2395e1-ddf5-4bd5-8730-ef76176cf426 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:37.741473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2395e1-ddf5-4bd5-8730-ef76176cf426 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:37.741739 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2395e1-ddf5-4bd5-8730-ef76176cf426 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:37.751395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2395e1-ddf5-4bd5-8730-ef76176cf426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-6c2395e1-ddf5-4bd5-8730-ef76176cf426 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:38:37.757097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:37.758308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:37.759474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:37.759474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:37.759474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:37.759474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:37.759474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:38:37.759474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:38:37.759474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:38:37.759474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:38:37.759474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:38:37.759474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:38:37.759474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:37.759474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:37.759474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 591/1156] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:38:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:37.764580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb023d0-43d0-4729-b1bc-1539efc7039f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:37.861238 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb023d0-43d0-4729-b1bc-1539efc7039f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:37.872645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:37.898065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c5c23-849b-41e7-8571-7db0bffa3129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:37.898711 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c5c23-849b-41e7-8571-7db0bffa3129 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:37.898975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c5c23-849b-41e7-8571-7db0bffa3129 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:37.899239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c5c23-849b-41e7-8571-7db0bffa3129 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:37.899937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c5c23-849b-41e7-8571-7db0bffa3129 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:37.900238 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c5c23-849b-41e7-8571-7db0bffa3129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:37.901591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c5c23-849b-41e7-8571-7db0bffa3129 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:37.909411 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 566/1157] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:38:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:37.915338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2395e1-ddf5-4bd5-8730-ef76176cf426 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:38.012604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2395e1-ddf5-4bd5-8730-ef76176cf426 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:38.023637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:38.046926 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991114c-ef22-4825-9e1c-3fe1630f0e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:38.047548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991114c-ef22-4825-9e1c-3fe1630f0e42 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:38.047828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991114c-ef22-4825-9e1c-3fe1630f0e42 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:38.048102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991114c-ef22-4825-9e1c-3fe1630f0e42 None admin] PATH_INFO: `/v3/users/4a73aa603af94c53ab054a5fd9a001d8` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:38.049247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991114c-ef22-4825-9e1c-3fe1630f0e42 None admin] RBAC: Authorizing `identity:get_user(user_id=4a73aa603af94c53ab054a5fd9a001d8)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:38.050746 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991114c-ef22-4825-9e1c-3fe1630f0e42 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:38.052360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 592/1158] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:37 2022] GET /identity/v3/users/4a73aa603af94c53ab054a5fd9a001d8 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:38.058158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c5c23-849b-41e7-8571-7db0bffa3129 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:38.156787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c5c23-849b-41e7-8571-7db0bffa3129 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:38.189465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795e914-3733-4a6c-8c8e-ae2b3947f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:38.190062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795e914-3733-4a6c-8c8e-ae2b3947f733 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:38.190336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795e914-3733-4a6c-8c8e-ae2b3947f733 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:38.190604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795e914-3733-4a6c-8c8e-ae2b3947f733 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:38.230078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:38.258597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795e914-3733-4a6c-8c8e-ae2b3947f733 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:38.260067 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795e914-3733-4a6c-8c8e-ae2b3947f733 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:38.354414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 567/1159] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:38:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:38:38.360297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991114c-ef22-4825-9e1c-3fe1630f0e42 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:38.458363 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991114c-ef22-4825-9e1c-3fe1630f0e42 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:38.492170 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7a8be-fdc3-48f5-bef0-32960f5b4860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:38.492806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7a8be-fdc3-48f5-bef0-32960f5b4860 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:38.493084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7a8be-fdc3-48f5-bef0-32960f5b4860 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:38.493355 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7a8be-fdc3-48f5-bef0-32960f5b4860 None admin] PATH_INFO: `/v3/projects/d752974e97c749be9f867eba56b1bbef` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:38.494502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7a8be-fdc3-48f5-bef0-32960f5b4860 None admin] RBAC: Authorizing `identity:get_project(project_id=d752974e97c749be9f867eba56b1bbef)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:38.495964 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7a8be-fdc3-48f5-bef0-32960f5b4860 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:38.497631 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 593/1160] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:38:38 2022] GET /identity/v3/projects/d752974e97c749be9f867eba56b1bbef => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:38.504683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795e914-3733-4a6c-8c8e-ae2b3947f733 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:38.573107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:38.602616 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795e914-3733-4a6c-8c8e-ae2b3947f733 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:38.636151 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede89ed9-2d83-4054-af8d-49390ce6f8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:38.636690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede89ed9-2d83-4054-af8d-49390ce6f8d2 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:38.636965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede89ed9-2d83-4054-af8d-49390ce6f8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:38.637230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede89ed9-2d83-4054-af8d-49390ce6f8d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:38.637997 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede89ed9-2d83-4054-af8d-49390ce6f8d2 None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=d752974e97c749be9f867eba56b1bbef, user.id=4a73aa603af94c53ab054a5fd9a001d8) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:38.638301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede89ed9-2d83-4054-af8d-49390ce6f8d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:38.639755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede89ed9-2d83-4054-af8d-49390ce6f8d2 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:38.645388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 568/1161] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:38:38 2022] GET /identity/v3/role_assignments?user.id=4a73aa603af94c53ab054a5fd9a001d8&scope.project.id=d752974e97c749be9f867eba56b1bbef&role.id=93eb204498fc49d297bcd1841e00818d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:40.685418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bfc1f-1c21-4621-b4cc-54292bb03f96 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:40.685984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bfc1f-1c21-4621-b4cc-54292bb03f96 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:40.685984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bfc1f-1c21-4621-b4cc-54292bb03f96 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:40.687331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 594/1162] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:40.693000 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589013b-7a5d-4bcc-a48a-1f5d22ffbd33 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:40.693412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589013b-7a5d-4bcc-a48a-1f5d22ffbd33 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:40.693570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589013b-7a5d-4bcc-a48a-1f5d22ffbd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:40.711514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-4589013b-7a5d-4bcc-a48a-1f5d22ffbd33 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:40.712676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-4589013b-7a5d-4bcc-a48a-1f5d22ffbd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:40.777092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 569/1163] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:40.799759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bfc1f-1c21-4621-b4cc-54292bb03f96 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:40.847486 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:40.902455 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bfc1f-1c21-4621-b4cc-54292bb03f96 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:40.934796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dee109-1b57-4082-be6c-aeeaf3445eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:40.935404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dee109-1b57-4082-be6c-aeeaf3445eb2 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:40.935682 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dee109-1b57-4082-be6c-aeeaf3445eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:40.935953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dee109-1b57-4082-be6c-aeeaf3445eb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:41.001051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dee109-1b57-4082-be6c-aeeaf3445eb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:41.002486 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dee109-1b57-4082-be6c-aeeaf3445eb2 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:41.046560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:41.100831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 595/1164] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:38:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:38:41.106858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589013b-7a5d-4bcc-a48a-1f5d22ffbd33 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:41.170214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:41.203709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589013b-7a5d-4bcc-a48a-1f5d22ffbd33 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:41.237014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a83333-3e48-436a-afa1-3839d37bcdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:41.237742 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a83333-3e48-436a-afa1-3839d37bcdcc None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:41.238023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a83333-3e48-436a-afa1-3839d37bcdcc None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:41.238293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a83333-3e48-436a-afa1-3839d37bcdcc None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:41.244807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a83333-3e48-436a-afa1-3839d37bcdcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:41.246232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a83333-3e48-436a-afa1-3839d37bcdcc None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-92a83333-3e48-436a-afa1-3839d37bcdcc None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:41.257507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:41.258801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:41.259675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 570/1165] 10.222.0.86 () {56 vars in 1117 bytes} [Fri Aug 5 19:38:41 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:41.264655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dee109-1b57-4082-be6c-aeeaf3445eb2 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:41.361601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dee109-1b57-4082-be6c-aeeaf3445eb2 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:41.393716 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc6640-97ae-4e22-823b-7d3d2d50a782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:41.394939 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc6640-97ae-4e22-823b-7d3d2d50a782 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:41.395218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc6640-97ae-4e22-823b-7d3d2d50a782 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:41.395490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc6640-97ae-4e22-823b-7d3d2d50a782 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:41.396173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc6640-97ae-4e22-823b-7d3d2d50a782 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:41.396482 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc6640-97ae-4e22-823b-7d3d2d50a782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:41.397838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc6640-97ae-4e22-823b-7d3d2d50a782 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:41.404917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 596/1166] 10.222.0.86 () {56 vars in 1115 bytes} [Fri Aug 5 19:38:41 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:43.419305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1de20-8a00-46ec-a0a0-b59893e7e292 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:43.419868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1de20-8a00-46ec-a0a0-b59893e7e292 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:43.419868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1de20-8a00-46ec-a0a0-b59893e7e292 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:43.421200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 571/1167] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:43.426528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e6022-da8b-441d-8643-cd3844edc541 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:43.427016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e6022-da8b-441d-8643-cd3844edc541 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:43.427118 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e6022-da8b-441d-8643-cd3844edc541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:43.444929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-f02e6022-da8b-441d-8643-cd3844edc541 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:43.446112 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f02e6022-da8b-441d-8643-cd3844edc541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:43.448922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:43.513243 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 597/1168] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:43.538019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1de20-8a00-46ec-a0a0-b59893e7e292 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:43.638267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1de20-8a00-46ec-a0a0-b59893e7e292 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:43.662851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:43.671989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36307244-064f-453b-9399-c823cef2c384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:43.672532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36307244-064f-453b-9399-c823cef2c384 None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:43.672805 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36307244-064f-453b-9399-c823cef2c384 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:43.673072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36307244-064f-453b-9399-c823cef2c384 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:43.673772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36307244-064f-453b-9399-c823cef2c384 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:43.675871 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36307244-064f-453b-9399-c823cef2c384 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:43.691874 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 572/1169] 10.222.0.86 () {60 vars in 1138 bytes} [Fri Aug 5 19:38:43 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:38:45.760760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae4479-d0b5-4798-bbc9-4cc1a2ae7192 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:45.761282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae4479-d0b5-4798-bbc9-4cc1a2ae7192 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:45.761282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae4479-d0b5-4798-bbc9-4cc1a2ae7192 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:45.762677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 598/1170] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:45.767774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69a637-a469-4a12-8b10-65706a4f9632 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:45.768176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69a637-a469-4a12-8b10-65706a4f9632 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:45.768327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69a637-a469-4a12-8b10-65706a4f9632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:45.789418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-df69a637-a469-4a12-8b10-65706a4f9632 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:45.790604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-df69a637-a469-4a12-8b10-65706a4f9632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:45.855869 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 573/1171] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:45.878864 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae4479-d0b5-4798-bbc9-4cc1a2ae7192 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:45.992582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae4479-d0b5-4798-bbc9-4cc1a2ae7192 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:46.025308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42e63e-7530-402c-b80a-a6170681564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:46.025859 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42e63e-7530-402c-b80a-a6170681564a None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:46.026132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42e63e-7530-402c-b80a-a6170681564a None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:46.026400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42e63e-7530-402c-b80a-a6170681564a None admin] PATH_INFO: `/v3/domains/db900e5b3fdb401c8c9cc7328e7e42d0` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:46.034967 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:46.037754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42e63e-7530-402c-b80a-a6170681564a None admin] RBAC: Authorizing `identity:get_domain(domain_id=db900e5b3fdb401c8c9cc7328e7e42d0)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:46.039413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42e63e-7530-402c-b80a-a6170681564a None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:46.041599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 599/1172] 10.222.0.86 () {56 vars in 1183 bytes} [Fri Aug 5 19:38:45 2022] GET /identity/v3/domains/db900e5b3fdb401c8c9cc7328e7e42d0 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:46.047119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69a637-a469-4a12-8b10-65706a4f9632 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:46.143423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69a637-a469-4a12-8b10-65706a4f9632 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:46.160655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:46.178025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df598f-5373-4c90-8aa1-6f026f34fe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:46.178646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df598f-5373-4c90-8aa1-6f026f34fe5e None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:46.178937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df598f-5373-4c90-8aa1-6f026f34fe5e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:46.179208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df598f-5373-4c90-8aa1-6f026f34fe5e None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:46.179893 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df598f-5373-4c90-8aa1-6f026f34fe5e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:46.181318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df598f-5373-4c90-8aa1-6f026f34fe5e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:46.208901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 574/1173] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:38:46 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:38:48.250591 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0ea9-f667-4429-bc06-81274843cc1e None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:48.251174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0ea9-f667-4429-bc06-81274843cc1e None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:48.251174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0ea9-f667-4429-bc06-81274843cc1e None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:48.252504 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 600/1174] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:48.258267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54737150-3456-411e-a35c-84672b239046 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:48.258814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54737150-3456-411e-a35c-84672b239046 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:48.258930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54737150-3456-411e-a35c-84672b239046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:48.281373 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-54737150-3456-411e-a35c-84672b239046 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:48.282571 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-54737150-3456-411e-a35c-84672b239046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:48.356932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 575/1175] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:48.381984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc0ea9-f667-4429-bc06-81274843cc1e None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:48.483215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc0ea9-f667-4429-bc06-81274843cc1e None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:48.516477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a007ef-6063-4a85-85db-3104d896b8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:48.517104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a007ef-6063-4a85-85db-3104d896b8d7 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:48.517375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a007ef-6063-4a85-85db-3104d896b8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:48.517640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a007ef-6063-4a85-85db-3104d896b8d7 None admin] PATH_INFO: `/v3/domains/db900e5b3fdb401c8c9cc7328e7e42d0` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:48.519223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a007ef-6063-4a85-85db-3104d896b8d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=db900e5b3fdb401c8c9cc7328e7e42d0)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:48.520668 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a007ef-6063-4a85-85db-3104d896b8d7 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:48.522249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 601/1176] 10.222.0.86 () {56 vars in 1183 bytes} [Fri Aug 5 19:38:48 2022] GET /identity/v3/domains/db900e5b3fdb401c8c9cc7328e7e42d0 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:48.528001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54737150-3456-411e-a35c-84672b239046 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:48.620763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:48.625512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54737150-3456-411e-a35c-84672b239046 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:48.660688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ad7d1-e997-416a-a889-b8a9ac1cc1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:48.661279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ad7d1-e997-416a-a889-b8a9ac1cc1d7 None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:48.661554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ad7d1-e997-416a-a889-b8a9ac1cc1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:48.661823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ad7d1-e997-416a-a889-b8a9ac1cc1d7 None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:48.662524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ad7d1-e997-416a-a889-b8a9ac1cc1d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:48.663975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ad7d1-e997-416a-a889-b8a9ac1cc1d7 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:48.685496 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 576/1177] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:38:48 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:38:50.736556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875afcf8-cb17-4164-93cd-48ea06d8b40c None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:50.737076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875afcf8-cb17-4164-93cd-48ea06d8b40c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:50.737076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875afcf8-cb17-4164-93cd-48ea06d8b40c None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:50.738356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 602/1178] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:50.744319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c294f5-c705-4bb0-8fdb-7330cc23dcb7 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:50.744745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c294f5-c705-4bb0-8fdb-7330cc23dcb7 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:50.744847 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c294f5-c705-4bb0-8fdb-7330cc23dcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:50.762882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c0c294f5-c705-4bb0-8fdb-7330cc23dcb7 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:50.764052 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c0c294f5-c705-4bb0-8fdb-7330cc23dcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:50.828971 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 577/1179] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:50.852170 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875afcf8-cb17-4164-93cd-48ea06d8b40c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:50.918882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:50.951765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875afcf8-cb17-4164-93cd-48ea06d8b40c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:50.984945 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebd4e3-c071-4e25-b5be-f141a1fdb928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:50.985562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebd4e3-c071-4e25-b5be-f141a1fdb928 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:50.985840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebd4e3-c071-4e25-b5be-f141a1fdb928 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:50.986110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebd4e3-c071-4e25-b5be-f141a1fdb928 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:50.996431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebd4e3-c071-4e25-b5be-f141a1fdb928 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-6cebd4e3-c071-4e25-b5be-f141a1fdb928 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:38:51.001191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:51.002289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:51.003423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:51.003423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:51.003423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:51.003423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:51.003423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:38:51.003423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:38:51.003423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:38:51.003423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:38:51.003423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:38:51.003423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:38:51.003423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:51.003423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:51.003423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 603/1180] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:38:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:51.008482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c294f5-c705-4bb0-8fdb-7330cc23dcb7 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:51.098164 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:51.115358 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c294f5-c705-4bb0-8fdb-7330cc23dcb7 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:51.149601 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2efff-fbcb-4b2e-9c90-08924e1ce99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:51.150214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2efff-fbcb-4b2e-9c90-08924e1ce99e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:51.150487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2efff-fbcb-4b2e-9c90-08924e1ce99e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:51.150785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2efff-fbcb-4b2e-9c90-08924e1ce99e None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:51.151494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2efff-fbcb-4b2e-9c90-08924e1ce99e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:51.151797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2efff-fbcb-4b2e-9c90-08924e1ce99e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:51.153160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2efff-fbcb-4b2e-9c90-08924e1ce99e None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:51.161587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 578/1181] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:38:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:51.167483 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebd4e3-c071-4e25-b5be-f141a1fdb928 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:51.265746 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebd4e3-c071-4e25-b5be-f141a1fdb928 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:51.292179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:51.300325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951902a9-72e2-4f98-ada1-5ad32fa8ce88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:51.301071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951902a9-72e2-4f98-ada1-5ad32fa8ce88 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:51.301381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951902a9-72e2-4f98-ada1-5ad32fa8ce88 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:51.301656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951902a9-72e2-4f98-ada1-5ad32fa8ce88 None admin] PATH_INFO: `/v3/users/ab0d09b513b44a539f5444a4e57ac1a7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:51.314752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951902a9-72e2-4f98-ada1-5ad32fa8ce88 None admin] RBAC: Authorizing `identity:get_user(user_id=ab0d09b513b44a539f5444a4e57ac1a7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:51.316734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951902a9-72e2-4f98-ada1-5ad32fa8ce88 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:51.318410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 604/1182] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:51 2022] GET /identity/v3/users/ab0d09b513b44a539f5444a4e57ac1a7 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:51.324497 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2efff-fbcb-4b2e-9c90-08924e1ce99e None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:51.421891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2efff-fbcb-4b2e-9c90-08924e1ce99e None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:51.454828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9237e57-06e7-4444-a207-d7489588f265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:51.455424 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9237e57-06e7-4444-a207-d7489588f265 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:51.455702 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9237e57-06e7-4444-a207-d7489588f265 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:51.456561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9237e57-06e7-4444-a207-d7489588f265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:51.471028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:51.524206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9237e57-06e7-4444-a207-d7489588f265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:51.525586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9237e57-06e7-4444-a207-d7489588f265 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:51.629666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 579/1183] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:38:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:38:51.635659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951902a9-72e2-4f98-ada1-5ad32fa8ce88 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:51.741159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951902a9-72e2-4f98-ada1-5ad32fa8ce88 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:51.777654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aef235-a649-486b-be06-3fa6d8ee87e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:51.778191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aef235-a649-486b-be06-3fa6d8ee87e1 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:51.778463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aef235-a649-486b-be06-3fa6d8ee87e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:51.778759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aef235-a649-486b-be06-3fa6d8ee87e1 None admin] PATH_INFO: `/v3/projects/49ec2272391841cdae2abc3771ffce6f` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:51.779906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aef235-a649-486b-be06-3fa6d8ee87e1 None admin] RBAC: Authorizing `identity:get_project(project_id=49ec2272391841cdae2abc3771ffce6f)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:51.781320 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aef235-a649-486b-be06-3fa6d8ee87e1 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:51.783034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 605/1184] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:38:51 2022] GET /identity/v3/projects/49ec2272391841cdae2abc3771ffce6f => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:51.790202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9237e57-06e7-4444-a207-d7489588f265 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:51.838249 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:51.890685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9237e57-06e7-4444-a207-d7489588f265 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:51.924111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639c6e7-6292-4318-a3bd-29d73651c998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:51.924757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639c6e7-6292-4318-a3bd-29d73651c998 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:51.925039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639c6e7-6292-4318-a3bd-29d73651c998 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:51.925307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639c6e7-6292-4318-a3bd-29d73651c998 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:51.926082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639c6e7-6292-4318-a3bd-29d73651c998 None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=49ec2272391841cdae2abc3771ffce6f, user.id=ab0d09b513b44a539f5444a4e57ac1a7) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:51.926386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639c6e7-6292-4318-a3bd-29d73651c998 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:51.927833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639c6e7-6292-4318-a3bd-29d73651c998 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:51.933171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 580/1185] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:38:51 2022] GET /identity/v3/role_assignments?user.id=ab0d09b513b44a539f5444a4e57ac1a7&scope.project.id=49ec2272391841cdae2abc3771ffce6f&role.id=93eb204498fc49d297bcd1841e00818d => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:53.938745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d9d31-7086-481b-a572-532b4cb0507c None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:53.939288 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d9d31-7086-481b-a572-532b4cb0507c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:53.939288 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d9d31-7086-481b-a572-532b4cb0507c None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:53.941422 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 606/1186] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:53.946801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25902f85-0cfe-413e-91ca-094837ac2f5d None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:53.947204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25902f85-0cfe-413e-91ca-094837ac2f5d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:53.947350 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25902f85-0cfe-413e-91ca-094837ac2f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:53.965180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-25902f85-0cfe-413e-91ca-094837ac2f5d None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:53.966353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-25902f85-0cfe-413e-91ca-094837ac2f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:54.032246 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 581/1187] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:54.055610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d9d31-7086-481b-a572-532b4cb0507c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:54.120327 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:54.173066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3d9d31-7086-481b-a572-532b4cb0507c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:54.209347 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26faac33-5dd0-4a32-80ec-a2c4d5fc68dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:54.209899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26faac33-5dd0-4a32-80ec-a2c4d5fc68dd None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:54.210174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26faac33-5dd0-4a32-80ec-a2c4d5fc68dd None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:54.210441 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26faac33-5dd0-4a32-80ec-a2c4d5fc68dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:54.220203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26faac33-5dd0-4a32-80ec-a2c4d5fc68dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-26faac33-5dd0-4a32-80ec-a2c4d5fc68dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:38:54.226140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:54.227301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:54.228299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:54.228299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:54.228299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:54.228299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:54.228299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:38:54.228299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:38:54.228299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:38:54.228299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:38:54.228299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:38:54.228299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:38:54.228299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:54.228299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:38:54.228299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 607/1188] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:38:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:54.233481 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25902f85-0cfe-413e-91ca-094837ac2f5d None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:54.274745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:54.332157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25902f85-0cfe-413e-91ca-094837ac2f5d None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:54.366882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed67634-ee96-4487-98b7-9b055912d391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:54.367495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed67634-ee96-4487-98b7-9b055912d391 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:54.367768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed67634-ee96-4487-98b7-9b055912d391 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:54.368037 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed67634-ee96-4487-98b7-9b055912d391 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:54.368835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed67634-ee96-4487-98b7-9b055912d391 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:54.369135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed67634-ee96-4487-98b7-9b055912d391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:54.370500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed67634-ee96-4487-98b7-9b055912d391 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:54.379104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 582/1189] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:38:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:54.385335 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26faac33-5dd0-4a32-80ec-a2c4d5fc68dd None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:54.484824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26faac33-5dd0-4a32-80ec-a2c4d5fc68dd None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:54.499465 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:54.524805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee7181-1831-4409-942c-bfe41fab17d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:54.525448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee7181-1831-4409-942c-bfe41fab17d5 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:54.525725 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee7181-1831-4409-942c-bfe41fab17d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:54.525996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee7181-1831-4409-942c-bfe41fab17d5 None admin] PATH_INFO: `/v3/users/ab0d09b513b44a539f5444a4e57ac1a7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:54.527225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee7181-1831-4409-942c-bfe41fab17d5 None admin] RBAC: Authorizing `identity:get_user(user_id=ab0d09b513b44a539f5444a4e57ac1a7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:54.528656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee7181-1831-4409-942c-bfe41fab17d5 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:54.530284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 608/1190] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:54 2022] GET /identity/v3/users/ab0d09b513b44a539f5444a4e57ac1a7 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:54.536901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed67634-ee96-4487-98b7-9b055912d391 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:54.640978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed67634-ee96-4487-98b7-9b055912d391 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:54.642904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:54.679216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957a468-4f9d-4893-9020-fec83675badd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:54.679848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957a468-4f9d-4893-9020-fec83675badd None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:54.680122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957a468-4f9d-4893-9020-fec83675badd None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:54.680396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957a468-4f9d-4893-9020-fec83675badd None admin] PATH_INFO: `/v3/projects/49ec2272391841cdae2abc3771ffce6f` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:54.681563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957a468-4f9d-4893-9020-fec83675badd None admin] RBAC: Authorizing `identity:get_project(project_id=49ec2272391841cdae2abc3771ffce6f)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:54.683106 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957a468-4f9d-4893-9020-fec83675badd None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:54.684777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 583/1191] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:38:54 2022] GET /identity/v3/projects/49ec2272391841cdae2abc3771ffce6f => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:54.691302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee7181-1831-4409-942c-bfe41fab17d5 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:54.796002 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee7181-1831-4409-942c-bfe41fab17d5 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:54.829535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08b12e-a16a-4c88-bb27-ddcaf06f8b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:54.830189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b12e-a16a-4c88-bb27-ddcaf06f8b0d None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:54.830465 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b12e-a16a-4c88-bb27-ddcaf06f8b0d None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:54.831287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b12e-a16a-4c88-bb27-ddcaf06f8b0d None admin] PATH_INFO: `/v3/projects/49ec2272391841cdae2abc3771ffce6f/users/ab0d09b513b44a539f5444a4e57ac1a7/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:54.833371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08b12e-a16a-4c88-bb27-ddcaf06f8b0d None admin] RBAC: Authorizing `identity:create_grant(project_id=49ec2272391841cdae2abc3771ffce6f, user_id=ab0d09b513b44a539f5444a4e57ac1a7, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:54.834840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08b12e-a16a-4c88-bb27-ddcaf06f8b0d None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:54.848842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 609/1192] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:38:54 2022] PUT /identity/v3/projects/49ec2272391841cdae2abc3771ffce6f/users/ab0d09b513b44a539f5444a4e57ac1a7/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:38:56.874361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e3e98-f063-483b-a59d-f7431116add0 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:56.875006 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e3e98-f063-483b-a59d-f7431116add0 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:56.875006 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e3e98-f063-483b-a59d-f7431116add0 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:56.876276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 584/1193] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:38:56.882066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f5e56-f4a1-4dba-9ae1-6d2a62a4f5da None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:56.882574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f5e56-f4a1-4dba-9ae1-6d2a62a4f5da None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:56.882722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f5e56-f4a1-4dba-9ae1-6d2a62a4f5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:56.901120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-d46f5e56-f4a1-4dba-9ae1-6d2a62a4f5da None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:38:56.902295 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d46f5e56-f4a1-4dba-9ae1-6d2a62a4f5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:38:56.938077 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:56.971013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 610/1194] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:38:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:38:56.997977 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e3e98-f063-483b-a59d-f7431116add0 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:57.102032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e3e98-f063-483b-a59d-f7431116add0 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:57.136557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b39a0-0ae2-4194-a6e6-4f16f2b2cc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:57.137096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b39a0-0ae2-4194-a6e6-4f16f2b2cc0a None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:57.137368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b39a0-0ae2-4194-a6e6-4f16f2b2cc0a None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:57.137632 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b39a0-0ae2-4194-a6e6-4f16f2b2cc0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:57.148310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b39a0-0ae2-4194-a6e6-4f16f2b2cc0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-919b39a0-0ae2-4194-a6e6-4f16f2b2cc0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:38:57.153051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:38:57.154149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:38:57.155135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:38:57.155135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:38:57.155135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:38:57.155135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:38:57.155135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:38:57.155135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:38:57.155135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:38:57.155135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:38:57.155135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:38:57.155135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:38:57.155135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:38:57.155135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:38:57.155135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 585/1195] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:38:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:38:57.160927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f5e56-f4a1-4dba-9ae1-6d2a62a4f5da None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:57.261843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f5e56-f4a1-4dba-9ae1-6d2a62a4f5da None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:57.295978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55680e24-8363-4b07-8d9e-70e350bbebea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:57.296584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55680e24-8363-4b07-8d9e-70e350bbebea None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:57.296861 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55680e24-8363-4b07-8d9e-70e350bbebea None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:57.297129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55680e24-8363-4b07-8d9e-70e350bbebea None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:57.297839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55680e24-8363-4b07-8d9e-70e350bbebea None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:57.298136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55680e24-8363-4b07-8d9e-70e350bbebea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:57.300114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55680e24-8363-4b07-8d9e-70e350bbebea None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:57.308251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 611/1196] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:38:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:57.315322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b39a0-0ae2-4194-a6e6-4f16f2b2cc0a None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:57.349434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:57.422049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b39a0-0ae2-4194-a6e6-4f16f2b2cc0a None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:57.457646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb2369-6546-4bcb-9c0b-123859dccb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:57.458271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb2369-6546-4bcb-9c0b-123859dccb26 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:57.458547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb2369-6546-4bcb-9c0b-123859dccb26 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:57.458846 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb2369-6546-4bcb-9c0b-123859dccb26 None admin] PATH_INFO: `/v3/users/ab0d09b513b44a539f5444a4e57ac1a7` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:57.460018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb2369-6546-4bcb-9c0b-123859dccb26 None admin] RBAC: Authorizing `identity:get_user(user_id=ab0d09b513b44a539f5444a4e57ac1a7)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:57.461441 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb2369-6546-4bcb-9c0b-123859dccb26 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:57.463089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 586/1197] 10.222.0.86 () {56 vars in 1177 bytes} [Fri Aug 5 19:38:57 2022] GET /identity/v3/users/ab0d09b513b44a539f5444a4e57ac1a7 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:57.469545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55680e24-8363-4b07-8d9e-70e350bbebea None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:57.508219 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:57.576228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55680e24-8363-4b07-8d9e-70e350bbebea None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:57.609343 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed9a8d-ea6d-4974-b03c-251e774cb987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:57.609952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed9a8d-ea6d-4974-b03c-251e774cb987 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:57.610225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed9a8d-ea6d-4974-b03c-251e774cb987 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:57.610497 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed9a8d-ea6d-4974-b03c-251e774cb987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:57.677035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed9a8d-ea6d-4974-b03c-251e774cb987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:57.678401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed9a8d-ea6d-4974-b03c-251e774cb987 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:57.710314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:57.776388 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 612/1198] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:38:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:38:57.782504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb2369-6546-4bcb-9c0b-123859dccb26 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:57.882715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:57.887392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb2369-6546-4bcb-9c0b-123859dccb26 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:57.921058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df016636-5788-4641-ad01-398d0b910bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:57.921684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df016636-5788-4641-ad01-398d0b910bf1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:57.921960 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df016636-5788-4641-ad01-398d0b910bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:57.922230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df016636-5788-4641-ad01-398d0b910bf1 None admin] PATH_INFO: `/v3/projects/49ec2272391841cdae2abc3771ffce6f` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:57.923408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df016636-5788-4641-ad01-398d0b910bf1 None admin] RBAC: Authorizing `identity:get_project(project_id=49ec2272391841cdae2abc3771ffce6f)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:57.924919 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df016636-5788-4641-ad01-398d0b910bf1 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:57.926757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 587/1199] 10.222.0.86 () {56 vars in 1186 bytes} [Fri Aug 5 19:38:57 2022] GET /identity/v3/projects/49ec2272391841cdae2abc3771ffce6f => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:38:57.933106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed9a8d-ea6d-4974-b03c-251e774cb987 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:38:58.039868 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed9a8d-ea6d-4974-b03c-251e774cb987 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:38:58.074142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c74872-e162-41cb-a0c8-4051d2660497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:38:58.074783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c74872-e162-41cb-a0c8-4051d2660497 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:38:58.075060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c74872-e162-41cb-a0c8-4051d2660497 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:38:58.075328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c74872-e162-41cb-a0c8-4051d2660497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:38:58.076105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c74872-e162-41cb-a0c8-4051d2660497 None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=49ec2272391841cdae2abc3771ffce6f, user.id=ab0d09b513b44a539f5444a4e57ac1a7) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:38:58.076405 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c74872-e162-41cb-a0c8-4051d2660497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:38:58.077818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c74872-e162-41cb-a0c8-4051d2660497 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:38:58.079581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:38:58.085976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 613/1200] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:38:57 2022] GET /identity/v3/role_assignments?user.id=ab0d09b513b44a539f5444a4e57ac1a7&scope.project.id=49ec2272391841cdae2abc3771ffce6f&role.id=93eb204498fc49d297bcd1841e00818d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:39:07.916859 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=17 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:39:08.086263 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:39:56.254787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d35d9-97cb-4a20-b830-2c9aa677c006 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:39:56.255470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d35d9-97cb-4a20-b830-2c9aa677c006 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:39:56.255875 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d35d9-97cb-4a20-b830-2c9aa677c006 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:39:56.258676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 588/1201] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:39:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:39:56.270193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37194207-4475-49cf-9c19-e203e68d21f2 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:39:56.270193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37194207-4475-49cf-9c19-e203e68d21f2 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:39:56.270193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37194207-4475-49cf-9c19-e203e68d21f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:39:56.305328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-37194207-4475-49cf-9c19-e203e68d21f2 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:39:56.306580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-37194207-4475-49cf-9c19-e203e68d21f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:39:56.425054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 614/1202] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:39:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:39:56.466435 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d35d9-97cb-4a20-b830-2c9aa677c006 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:39:56.619513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d35d9-97cb-4a20-b830-2c9aa677c006 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:39:56.663310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46490a47-c015-4691-857d-aaac11f9c2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:39:56.664072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46490a47-c015-4691-857d-aaac11f9c2ec None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:39:56.664410 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46490a47-c015-4691-857d-aaac11f9c2ec None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:39:56.664734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46490a47-c015-4691-857d-aaac11f9c2ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:39:56.677599 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46490a47-c015-4691-857d-aaac11f9c2ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:39:56.679193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46490a47-c015-4691-857d-aaac11f9c2ec None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-46490a47-c015-4691-857d-aaac11f9c2ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:39:56.697962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:39:56.699310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:39:56.700289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:39:56.700289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:39:56.700289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:39:56.700289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:39:56.700289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 589/1203] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:39:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:39:56.709960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37194207-4475-49cf-9c19-e203e68d21f2 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:39:56.872259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37194207-4475-49cf-9c19-e203e68d21f2 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:39:56.920450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:39:56.929700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e9970-f6bd-4a8c-9566-4b98f09fabcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:39:56.930640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e9970-f6bd-4a8c-9566-4b98f09fabcf None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:39:56.931076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e9970-f6bd-4a8c-9566-4b98f09fabcf None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:39:56.931534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e9970-f6bd-4a8c-9566-4b98f09fabcf None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:39:56.932394 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e9970-f6bd-4a8c-9566-4b98f09fabcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:39:56.932851 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e9970-f6bd-4a8c-9566-4b98f09fabcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:39:56.935046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e9970-f6bd-4a8c-9566-4b98f09fabcf None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:39:56.961928 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 615/1204] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:39:56.968392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46490a47-c015-4691-857d-aaac11f9c2ec None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:39:57.067099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:39:57.142057 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46490a47-c015-4691-857d-aaac11f9c2ec None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:39:57.191765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3440080-e963-4844-9047-6f65643e6e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:39:57.193052 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3440080-e963-4844-9047-6f65643e6e90 None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:39:57.193843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3440080-e963-4844-9047-6f65643e6e90 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:39:57.194802 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3440080-e963-4844-9047-6f65643e6e90 None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:39:57.196200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3440080-e963-4844-9047-6f65643e6e90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:39:57.197884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3440080-e963-4844-9047-6f65643e6e90 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:39:57.226767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 590/1205] 10.222.0.86 () {60 vars in 1132 bytes} [Fri Aug 5 19:39:56 2022] POST /identity/v3/users => generated 251 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:40:00.172820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c2829-e3fb-4e56-af09-5ce17c256e26 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:00.173453 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c2829-e3fb-4e56-af09-5ce17c256e26 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:00.173684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c2829-e3fb-4e56-af09-5ce17c256e26 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:00.176183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 616/1206] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:40:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:40:00.183819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bde28-e818-4e00-9541-225c7a1ca17e None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:00.184526 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bde28-e818-4e00-9541-225c7a1ca17e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:00.184526 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bde28-e818-4e00-9541-225c7a1ca17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:00.211598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-a08bde28-e818-4e00-9541-225c7a1ca17e None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:40:00.213504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a08bde28-e818-4e00-9541-225c7a1ca17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:40:00.323166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 591/1207] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:40:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:40:00.372103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c2829-e3fb-4e56-af09-5ce17c256e26 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:00.520935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c2829-e3fb-4e56-af09-5ce17c256e26 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:00.562390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac61bf4-2362-429f-b3d2-9b7d63894240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:00.563741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac61bf4-2362-429f-b3d2-9b7d63894240 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:00.564040 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac61bf4-2362-429f-b3d2-9b7d63894240 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:00.564315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac61bf4-2362-429f-b3d2-9b7d63894240 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:00.576474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac61bf4-2362-429f-b3d2-9b7d63894240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-4ac61bf4-2362-429f-b3d2-9b7d63894240 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:40:00.582958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:00.584873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:00.585731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:00.585731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:00.585731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:00.585731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:00.585731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:40:00.585731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:40:00.585731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:40:00.585731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:40:00.585731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:40:00.585731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:40:00.585731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:40:00.585731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:00.585731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 617/1208] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:40:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:40:00.591196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08bde28-e818-4e00-9541-225c7a1ca17e None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:00.649493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:00.743756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08bde28-e818-4e00-9541-225c7a1ca17e None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:00.803711 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8c4db-f962-4f30-930c-5a90a00b6793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:00.804784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8c4db-f962-4f30-930c-5a90a00b6793 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:00.805212 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8c4db-f962-4f30-930c-5a90a00b6793 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:00.805685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8c4db-f962-4f30-930c-5a90a00b6793 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:00.806980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8c4db-f962-4f30-930c-5a90a00b6793 None admin] RBAC: Adding query filter params (name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:00.807491 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8c4db-f962-4f30-930c-5a90a00b6793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:00.809910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b8c4db-f962-4f30-930c-5a90a00b6793 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:00.829146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 592/1209] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:40:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:00.830581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac61bf4-2362-429f-b3d2-9b7d63894240 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:00.900653 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:00.972284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac61bf4-2362-429f-b3d2-9b7d63894240 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:01.014123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e694873-c4a4-46fd-8b82-143bb4b5ec5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:01.014690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e694873-c4a4-46fd-8b82-143bb4b5ec5f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:01.014975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e694873-c4a4-46fd-8b82-143bb4b5ec5f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:01.015251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e694873-c4a4-46fd-8b82-143bb4b5ec5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:01.030643 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e694873-c4a4-46fd-8b82-143bb4b5ec5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:01.032138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e694873-c4a4-46fd-8b82-143bb4b5ec5f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-9e694873-c4a4-46fd-8b82-143bb4b5ec5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:40:01.052224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:01.053401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:01.055174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:40:01.055174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:40:01.055174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:01.055174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:01.055174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 618/1210] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:01.063933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8c4db-f962-4f30-930c-5a90a00b6793 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:01.213779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:01.217559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8c4db-f962-4f30-930c-5a90a00b6793 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:01.270766 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca898b67-4f5e-4fa0-a2ef-10b9ac2e6f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:01.271722 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca898b67-4f5e-4fa0-a2ef-10b9ac2e6f61 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:01.272183 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca898b67-4f5e-4fa0-a2ef-10b9ac2e6f61 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:01.272663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca898b67-4f5e-4fa0-a2ef-10b9ac2e6f61 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:01.273440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca898b67-4f5e-4fa0-a2ef-10b9ac2e6f61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:01.273869 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca898b67-4f5e-4fa0-a2ef-10b9ac2e6f61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:01.275648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca898b67-4f5e-4fa0-a2ef-10b9ac2e6f61 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:01.294691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 593/1211] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:01.300040 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e694873-c4a4-46fd-8b82-143bb4b5ec5f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:01.429265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e694873-c4a4-46fd-8b82-143bb4b5ec5f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:01.431336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:01.475123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c30a04-3147-4a9e-88d8-18faa8380f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:01.476308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30a04-3147-4a9e-88d8-18faa8380f1f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:01.476825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30a04-3147-4a9e-88d8-18faa8380f1f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:01.477374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c30a04-3147-4a9e-88d8-18faa8380f1f None admin] PATH_INFO: `/v3/users/placement` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:01.491657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c30a04-3147-4a9e-88d8-18faa8380f1f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:01.493986 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c30a04-3147-4a9e-88d8-18faa8380f1f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-d0c30a04-3147-4a9e-88d8-18faa8380f1f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:40:01.515410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:01.517600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:01.518603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:01.518603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:01.518603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:40:01.518603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:40:01.518603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:40:01.518603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:40:01.518603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:40:01.518603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:40:01.518603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:40:01.518603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:01.518603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 619/1212] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:40:01.525115 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca898b67-4f5e-4fa0-a2ef-10b9ac2e6f61 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:01.680142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca898b67-4f5e-4fa0-a2ef-10b9ac2e6f61 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:01.732667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fd326-a4b5-42f0-b4bd-ba014dbd1f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:01.733436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fd326-a4b5-42f0-b4bd-ba014dbd1f17 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:01.733940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fd326-a4b5-42f0-b4bd-ba014dbd1f17 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:01.734075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fd326-a4b5-42f0-b4bd-ba014dbd1f17 None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:01.736111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fd326-a4b5-42f0-b4bd-ba014dbd1f17 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:01.736111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fd326-a4b5-42f0-b4bd-ba014dbd1f17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:01.738106 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fd326-a4b5-42f0-b4bd-ba014dbd1f17 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:01.767989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 594/1213] 10.222.0.86 () {56 vars in 1143 bytes} [Fri Aug 5 19:40:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:01.777109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c30a04-3147-4a9e-88d8-18faa8380f1f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:01.934931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c30a04-3147-4a9e-88d8-18faa8380f1f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:01.982498 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a6c20-7f90-4e9a-83a1-9fd3a7b09a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:01.983161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a6c20-7f90-4e9a-83a1-9fd3a7b09a15 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:01.983444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a6c20-7f90-4e9a-83a1-9fd3a7b09a15 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:01.983717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a6c20-7f90-4e9a-83a1-9fd3a7b09a15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:02.024182 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:02.074984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a6c20-7f90-4e9a-83a1-9fd3a7b09a15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:02.076379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a6c20-7f90-4e9a-83a1-9fd3a7b09a15 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:02.205113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 620/1214] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:40:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:40:02.214482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fd326-a4b5-42f0-b4bd-ba014dbd1f17 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:02.239393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:02.351729 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fd326-a4b5-42f0-b4bd-ba014dbd1f17 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:02.403898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e740b77-d498-48b6-9a95-dc2455d80ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:02.404801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e740b77-d498-48b6-9a95-dc2455d80ccc None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:02.405161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e740b77-d498-48b6-9a95-dc2455d80ccc None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:02.405678 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e740b77-d498-48b6-9a95-dc2455d80ccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:02.422611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e740b77-d498-48b6-9a95-dc2455d80ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:02.424788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e740b77-d498-48b6-9a95-dc2455d80ccc None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-9e740b77-d498-48b6-9a95-dc2455d80ccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:40:02.444932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:02.446148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:02.447193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:40:02.447193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:40:02.447193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:02.447193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:02.447193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 595/1215] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:02.452098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a6c20-7f90-4e9a-83a1-9fd3a7b09a15 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:02.535046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:02.586972 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a6c20-7f90-4e9a-83a1-9fd3a7b09a15 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:02.650415 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec34cf-7650-46a6-88a7-0339cb2d066b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:02.651187 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec34cf-7650-46a6-88a7-0339cb2d066b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:02.651645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec34cf-7650-46a6-88a7-0339cb2d066b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:02.652256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbec34cf-7650-46a6-88a7-0339cb2d066b None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:02.653632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec34cf-7650-46a6-88a7-0339cb2d066b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:02.654240 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec34cf-7650-46a6-88a7-0339cb2d066b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:02.657265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbec34cf-7650-46a6-88a7-0339cb2d066b None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:02.678902 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 621/1216] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:40:02.689672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e740b77-d498-48b6-9a95-dc2455d80ccc None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:02.779597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:02.816930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e740b77-d498-48b6-9a95-dc2455d80ccc None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:02.867340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edcad7-2d86-4e60-a4d8-ebcda63a8b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:02.868011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edcad7-2d86-4e60-a4d8-ebcda63a8b70 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:02.868332 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edcad7-2d86-4e60-a4d8-ebcda63a8b70 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:02.868606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edcad7-2d86-4e60-a4d8-ebcda63a8b70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:02.876940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86edcad7-2d86-4e60-a4d8-ebcda63a8b70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:02.878508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86edcad7-2d86-4e60-a4d8-ebcda63a8b70 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-86edcad7-2d86-4e60-a4d8-ebcda63a8b70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:40:02.893825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:02.896026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:02.897019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:40:02.897019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:40:02.897019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:40:02.897019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:40:02.897019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:40:02.897019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:40:02.897019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:40:02.897019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:02.897019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 596/1217] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:40:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:02.901621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec34cf-7650-46a6-88a7-0339cb2d066b None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:03.045534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbec34cf-7650-46a6-88a7-0339cb2d066b None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:03.090594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:03.110348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3343d-d1f3-42ce-9472-f42893d03256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:03.111209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3343d-d1f3-42ce-9472-f42893d03256 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:03.111560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3343d-d1f3-42ce-9472-f42893d03256 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:03.111979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3343d-d1f3-42ce-9472-f42893d03256 None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:03.113220 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3343d-d1f3-42ce-9472-f42893d03256 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:03.113755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3343d-d1f3-42ce-9472-f42893d03256 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:03.116420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3343d-d1f3-42ce-9472-f42893d03256 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:03.134242 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 622/1218] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:40:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:40:03.141773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edcad7-2d86-4e60-a4d8-ebcda63a8b70 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:03.290351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edcad7-2d86-4e60-a4d8-ebcda63a8b70 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:03.331623 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:03.340551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890a71b-20e3-406b-aedd-8e8bb3a7b655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:03.341363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890a71b-20e3-406b-aedd-8e8bb3a7b655 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:03.341778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890a71b-20e3-406b-aedd-8e8bb3a7b655 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:03.342167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890a71b-20e3-406b-aedd-8e8bb3a7b655 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:03.343160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890a71b-20e3-406b-aedd-8e8bb3a7b655 None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=cce71f93c80d433b830c448d0a7a14e3) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:03.343589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890a71b-20e3-406b-aedd-8e8bb3a7b655 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:03.345737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890a71b-20e3-406b-aedd-8e8bb3a7b655 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:03.352325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 597/1219] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:40:03 2022] GET /identity/v3/role_assignments?user.id=cce71f93c80d433b830c448d0a7a14e3&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:05.551709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5216e-6ffc-4292-8cad-841a52819f02 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:05.552313 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5216e-6ffc-4292-8cad-841a52819f02 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:05.552313 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5216e-6ffc-4292-8cad-841a52819f02 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:05.553861 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 623/1220] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:40:05.559481 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932a551-cd0a-4f9f-89b4-b01fb36ae10b None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:05.560024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932a551-cd0a-4f9f-89b4-b01fb36ae10b None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:05.560024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932a551-cd0a-4f9f-89b4-b01fb36ae10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:05.580573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-0932a551-cd0a-4f9f-89b4-b01fb36ae10b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:40:05.581771 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-0932a551-cd0a-4f9f-89b4-b01fb36ae10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:40:05.666820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 598/1221] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:40:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:40:05.700830 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5216e-6ffc-4292-8cad-841a52819f02 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:05.828588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5216e-6ffc-4292-8cad-841a52819f02 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:05.875636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873761f-982a-43e4-9a90-d8f651e46be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:05.876418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873761f-982a-43e4-9a90-d8f651e46be1 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:05.876692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873761f-982a-43e4-9a90-d8f651e46be1 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:05.877020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873761f-982a-43e4-9a90-d8f651e46be1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:05.891573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873761f-982a-43e4-9a90-d8f651e46be1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-8873761f-982a-43e4-9a90-d8f651e46be1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:40:05.898395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:05.900768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:05.902401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:05.902401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:05.902401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:05.902401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:05.902401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:40:05.902401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:40:05.902401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:40:05.902401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:40:05.902401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:40:05.902401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:40:05.902401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:40:05.902401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:05.902401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 624/1222] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:40:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:05.907393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932a551-cd0a-4f9f-89b4-b01fb36ae10b None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:06.040951 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932a551-cd0a-4f9f-89b4-b01fb36ae10b None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:06.074566 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:06.088573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6f952-d668-4da4-8dc9-9abffccb2724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:06.089137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6f952-d668-4da4-8dc9-9abffccb2724 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:06.089417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6f952-d668-4da4-8dc9-9abffccb2724 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:06.089688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6f952-d668-4da4-8dc9-9abffccb2724 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:06.091237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6f952-d668-4da4-8dc9-9abffccb2724 None admin] RBAC: Adding query filter params (name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:06.091555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6f952-d668-4da4-8dc9-9abffccb2724 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:06.093149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6f952-d668-4da4-8dc9-9abffccb2724 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:06.107857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 599/1223] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:40:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:06.109956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873761f-982a-43e4-9a90-d8f651e46be1 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:06.175404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:06.253851 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873761f-982a-43e4-9a90-d8f651e46be1 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:06.306344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2aff9f-36c2-4f19-a763-cf5937caabf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:06.307056 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2aff9f-36c2-4f19-a763-cf5937caabf3 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:06.307352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2aff9f-36c2-4f19-a763-cf5937caabf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:06.307632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2aff9f-36c2-4f19-a763-cf5937caabf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:06.320724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2aff9f-36c2-4f19-a763-cf5937caabf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:06.322980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2aff9f-36c2-4f19-a763-cf5937caabf3 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-6a2aff9f-36c2-4f19-a763-cf5937caabf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:40:06.348218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:06.349370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:06.350401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:40:06.350401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:40:06.350401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:06.350401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:06.350401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 625/1224] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:40:06.356834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6f952-d668-4da4-8dc9-9abffccb2724 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:06.503629 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6f952-d668-4da4-8dc9-9abffccb2724 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:06.542330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74260f-a069-4219-b6f0-0e901237d127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:06.542980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74260f-a069-4219-b6f0-0e901237d127 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:06.543232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74260f-a069-4219-b6f0-0e901237d127 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:06.543511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74260f-a069-4219-b6f0-0e901237d127 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:06.544201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74260f-a069-4219-b6f0-0e901237d127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:06.544504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74260f-a069-4219-b6f0-0e901237d127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:06.545877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74260f-a069-4219-b6f0-0e901237d127 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:06.558201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 600/1225] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:06.564870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2aff9f-36c2-4f19-a763-cf5937caabf3 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:06.693955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:06.700719 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2aff9f-36c2-4f19-a763-cf5937caabf3 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:06.748833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fd3e4-b91c-4dac-89e1-a28be7198096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:06.750736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fd3e4-b91c-4dac-89e1-a28be7198096 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:06.751197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fd3e4-b91c-4dac-89e1-a28be7198096 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:06.751671 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fd3e4-b91c-4dac-89e1-a28be7198096 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:06.765311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fd3e4-b91c-4dac-89e1-a28be7198096 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:06.768748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fd3e4-b91c-4dac-89e1-a28be7198096 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-398fd3e4-b91c-4dac-89e1-a28be7198096 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:40:06.786472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:06.788454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:06.790330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:06.790330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:06.790330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:40:06.790330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:40:06.790330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:40:06.790330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:40:06.790330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:40:06.790330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:40:06.790330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:40:06.790330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:06.790330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 626/1226] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:06.798301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74260f-a069-4219-b6f0-0e901237d127 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:06.861914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:06.942900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74260f-a069-4219-b6f0-0e901237d127 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:06.986957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf68394-5263-44d0-ba29-6c8096c82e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:06.987632 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf68394-5263-44d0-ba29-6c8096c82e4f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:06.987994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf68394-5263-44d0-ba29-6c8096c82e4f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:06.988264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf68394-5263-44d0-ba29-6c8096c82e4f None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:06.989054 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf68394-5263-44d0-ba29-6c8096c82e4f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:06.989354 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf68394-5263-44d0-ba29-6c8096c82e4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:06.990815 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf68394-5263-44d0-ba29-6c8096c82e4f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:07.015563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 601/1227] 10.222.0.86 () {56 vars in 1143 bytes} [Fri Aug 5 19:40:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:07.017222 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fd3e4-b91c-4dac-89e1-a28be7198096 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:07.139962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fd3e4-b91c-4dac-89e1-a28be7198096 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:07.183437 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9aeafb-6732-435c-97af-d63a98f62a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:07.184147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9aeafb-6732-435c-97af-d63a98f62a81 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:07.184528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9aeafb-6732-435c-97af-d63a98f62a81 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:07.184828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9aeafb-6732-435c-97af-d63a98f62a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:07.197619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9aeafb-6732-435c-97af-d63a98f62a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:07.199264 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9aeafb-6732-435c-97af-d63a98f62a81 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-9e9aeafb-6732-435c-97af-d63a98f62a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:40:07.217013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:07.218228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:07.220064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:40:07.220064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:40:07.220064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:07.220064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:07.220064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 627/1228] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:07.226766 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf68394-5263-44d0-ba29-6c8096c82e4f None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:07.355947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf68394-5263-44d0-ba29-6c8096c82e4f None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:07.359309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:07.401877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2433e42-cf04-4129-98ec-ace9bad20eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:07.402497 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2433e42-cf04-4129-98ec-ace9bad20eb3 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:07.402792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2433e42-cf04-4129-98ec-ace9bad20eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:07.403070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2433e42-cf04-4129-98ec-ace9bad20eb3 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:07.403884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2433e42-cf04-4129-98ec-ace9bad20eb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:07.404190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2433e42-cf04-4129-98ec-ace9bad20eb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:07.405573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2433e42-cf04-4129-98ec-ace9bad20eb3 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:07.417726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 602/1229] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:07.426209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9aeafb-6732-435c-97af-d63a98f62a81 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:07.438706 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:07.546418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9aeafb-6732-435c-97af-d63a98f62a81 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:07.584472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e05fd-5ba6-4b32-95a7-cc4786c1f337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:07.585174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e05fd-5ba6-4b32-95a7-cc4786c1f337 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:07.585515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e05fd-5ba6-4b32-95a7-cc4786c1f337 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:07.585841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e05fd-5ba6-4b32-95a7-cc4786c1f337 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:07.597051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e05fd-5ba6-4b32-95a7-cc4786c1f337 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:07.599804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e05fd-5ba6-4b32-95a7-cc4786c1f337 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-d66e05fd-5ba6-4b32-95a7-cc4786c1f337 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:40:07.615068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:07.616079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:07.617016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:40:07.617016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:40:07.617016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:40:07.617016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:40:07.617016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:40:07.617016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:40:07.617016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:40:07.617016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:07.617016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 628/1230] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:40:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:07.624853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2433e42-cf04-4129-98ec-ace9bad20eb3 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:07.750883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2433e42-cf04-4129-98ec-ace9bad20eb3 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:07.795286 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6ba81-50e4-4b57-a094-b74a25659e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:07.796139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6ba81-50e4-4b57-a094-b74a25659e82 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:07.796422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6ba81-50e4-4b57-a094-b74a25659e82 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:07.796732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6ba81-50e4-4b57-a094-b74a25659e82 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:07.797835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6ba81-50e4-4b57-a094-b74a25659e82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:07.798133 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6ba81-50e4-4b57-a094-b74a25659e82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:07.799892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6ba81-50e4-4b57-a094-b74a25659e82 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:07.816218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 603/1231] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:40:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:07.823286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e05fd-5ba6-4b32-95a7-cc4786c1f337 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:07.922446 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:07.981064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e05fd-5ba6-4b32-95a7-cc4786c1f337 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:08.026815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa30065-1c6b-4be3-8f5f-2e7952ed21fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:08.027479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa30065-1c6b-4be3-8f5f-2e7952ed21fa None admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:08.027753 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa30065-1c6b-4be3-8f5f-2e7952ed21fa None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:08.028033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa30065-1c6b-4be3-8f5f-2e7952ed21fa None admin] PATH_INFO: `/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/cce71f93c80d433b830c448d0a7a14e3/roles/02001b18565340ce984343c5d36daaf9` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:08.045464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa30065-1c6b-4be3-8f5f-2e7952ed21fa None admin] RBAC: Authorizing `identity:create_grant(project_id=9c73d2cfaa1c47829ca6936186d2bb26, user_id=cce71f93c80d433b830c448d0a7a14e3, role_id=02001b18565340ce984343c5d36daaf9)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:08.047001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa30065-1c6b-4be3-8f5f-2e7952ed21fa None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:08.061632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 629/1232] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:40:07 2022] PUT /identity/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/cce71f93c80d433b830c448d0a7a14e3/roles/02001b18565340ce984343c5d36daaf9 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:40:10.449806 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180010d-a247-457c-a6d7-5c2fb152e83f None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:10.450463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180010d-a247-457c-a6d7-5c2fb152e83f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:10.450463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180010d-a247-457c-a6d7-5c2fb152e83f None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:10.451924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 604/1233] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:40:10.458600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d049b-8915-472f-9570-1af4d0830adf None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:10.459485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d049b-8915-472f-9570-1af4d0830adf None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:10.459485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d049b-8915-472f-9570-1af4d0830adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:10.480614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-d01d049b-8915-472f-9570-1af4d0830adf None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:40:10.482649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d01d049b-8915-472f-9570-1af4d0830adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:40:10.568924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 630/1234] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:40:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:40:10.608885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180010d-a247-457c-a6d7-5c2fb152e83f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:10.640557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:10.739122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180010d-a247-457c-a6d7-5c2fb152e83f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:10.782298 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f9a49-6cb7-4cef-9522-3594551ac82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:10.782298 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f9a49-6cb7-4cef-9522-3594551ac82b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:10.782636 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f9a49-6cb7-4cef-9522-3594551ac82b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:10.782861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f9a49-6cb7-4cef-9522-3594551ac82b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:10.794422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f9a49-6cb7-4cef-9522-3594551ac82b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-349f9a49-6cb7-4cef-9522-3594551ac82b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:40:10.800267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:10.801978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:10.803313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:10.803313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:10.803313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:10.803313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:10.803313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:40:10.803313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:40:10.803313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:40:10.803313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:40:10.803313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:40:10.803313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:40:10.803313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:40:10.803313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:10.803313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 605/1235] 10.222.0.86 () {56 vars in 1102 bytes} [Fri Aug 5 19:40:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:40:10.809318 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d049b-8915-472f-9570-1af4d0830adf None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:10.873162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:10.944547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d049b-8915-472f-9570-1af4d0830adf None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:10.984651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5476b90-3fef-4758-b582-bbae0c586638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:10.985276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5476b90-3fef-4758-b582-bbae0c586638 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:10.985702 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5476b90-3fef-4758-b582-bbae0c586638 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:10.985976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5476b90-3fef-4758-b582-bbae0c586638 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:10.986735 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5476b90-3fef-4758-b582-bbae0c586638 None admin] RBAC: Adding query filter params (name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:10.987054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5476b90-3fef-4758-b582-bbae0c586638 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:10.988620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5476b90-3fef-4758-b582-bbae0c586638 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:11.001553 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 631/1236] 10.222.0.86 () {56 vars in 1103 bytes} [Fri Aug 5 19:40:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:11.009520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f9a49-6cb7-4cef-9522-3594551ac82b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:11.119359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:11.139426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f9a49-6cb7-4cef-9522-3594551ac82b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:11.198139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a2fbe-27f6-4a7b-be3d-3064914a21c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:11.199117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a2fbe-27f6-4a7b-be3d-3064914a21c9 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:11.199537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a2fbe-27f6-4a7b-be3d-3064914a21c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:11.199911 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a2fbe-27f6-4a7b-be3d-3064914a21c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:11.217864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a2fbe-27f6-4a7b-be3d-3064914a21c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:11.221283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a2fbe-27f6-4a7b-be3d-3064914a21c9 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-db9a2fbe-27f6-4a7b-be3d-3064914a21c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:40:11.243289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:11.244425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:11.245410 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:40:11.245410 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:40:11.245410 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:11.245410 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:11.245410 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 606/1237] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:11.254384 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5476b90-3fef-4758-b582-bbae0c586638 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:11.369731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5476b90-3fef-4758-b582-bbae0c586638 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:11.381259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:11.411344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324b4db-cf1a-4371-987a-769d13a86389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:11.411969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324b4db-cf1a-4371-987a-769d13a86389 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:11.412243 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324b4db-cf1a-4371-987a-769d13a86389 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:11.412511 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324b4db-cf1a-4371-987a-769d13a86389 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:11.413216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324b4db-cf1a-4371-987a-769d13a86389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:11.413518 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324b4db-cf1a-4371-987a-769d13a86389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:11.414954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324b4db-cf1a-4371-987a-769d13a86389 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:11.428039 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 632/1238] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:40:11.434602 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a2fbe-27f6-4a7b-be3d-3064914a21c9 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:11.568779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a2fbe-27f6-4a7b-be3d-3064914a21c9 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:11.608453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab01fa3-e8c0-410c-9369-382e94b96155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:11.609886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab01fa3-e8c0-410c-9369-382e94b96155 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:11.610469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab01fa3-e8c0-410c-9369-382e94b96155 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:11.610469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab01fa3-e8c0-410c-9369-382e94b96155 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:11.613395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:11.619311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab01fa3-e8c0-410c-9369-382e94b96155 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:11.621118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab01fa3-e8c0-410c-9369-382e94b96155 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-bab01fa3-e8c0-410c-9369-382e94b96155 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:40:11.633691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:11.635162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:11.636416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:11.636416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:11.636416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:40:11.636416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:40:11.636416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:40:11.636416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:40:11.636416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:40:11.636416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:40:11.636416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:40:11.636416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:11.636416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 607/1239] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:11.641600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324b4db-cf1a-4371-987a-769d13a86389 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:11.773990 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324b4db-cf1a-4371-987a-769d13a86389 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:11.822561 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d91e9f-19a7-4016-949f-068ea8ca78eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:11.823390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d91e9f-19a7-4016-949f-068ea8ca78eb None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:11.823785 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d91e9f-19a7-4016-949f-068ea8ca78eb None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:11.824379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d91e9f-19a7-4016-949f-068ea8ca78eb None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:11.825423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d91e9f-19a7-4016-949f-068ea8ca78eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:11.825823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d91e9f-19a7-4016-949f-068ea8ca78eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:11.828087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d91e9f-19a7-4016-949f-068ea8ca78eb None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:11.848895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 633/1240] 10.222.0.86 () {56 vars in 1143 bytes} [Fri Aug 5 19:40:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:11.856332 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab01fa3-e8c0-410c-9369-382e94b96155 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:12.009147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab01fa3-e8c0-410c-9369-382e94b96155 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:12.053998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4067b55-519a-4fa3-b143-21ba7b51ce73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:12.054779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4067b55-519a-4fa3-b143-21ba7b51ce73 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:12.055191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4067b55-519a-4fa3-b143-21ba7b51ce73 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:12.055640 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4067b55-519a-4fa3-b143-21ba7b51ce73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:12.144471 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:12.153990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4067b55-519a-4fa3-b143-21ba7b51ce73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:12.155488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4067b55-519a-4fa3-b143-21ba7b51ce73 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:12.277684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 608/1241] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:40:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:40:12.278984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d91e9f-19a7-4016-949f-068ea8ca78eb None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:12.308624 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:12.393644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d91e9f-19a7-4016-949f-068ea8ca78eb None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:12.434007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc04f18-57fc-49f7-bf73-3c91e38f0a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:12.434730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc04f18-57fc-49f7-bf73-3c91e38f0a40 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:12.435083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc04f18-57fc-49f7-bf73-3c91e38f0a40 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:12.435505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc04f18-57fc-49f7-bf73-3c91e38f0a40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:12.448164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc04f18-57fc-49f7-bf73-3c91e38f0a40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:12.449905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc04f18-57fc-49f7-bf73-3c91e38f0a40 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-abc04f18-57fc-49f7-bf73-3c91e38f0a40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:40:12.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:12.470051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:12.471279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:40:12.471279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:40:12.471279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:12.471279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:12.471279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 634/1242] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:12.476636 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4067b55-519a-4fa3-b143-21ba7b51ce73 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:12.591756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:12.599342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4067b55-519a-4fa3-b143-21ba7b51ce73 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:12.647180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc3f77-b5e6-4e72-9cb9-7dd076511208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:12.648002 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc3f77-b5e6-4e72-9cb9-7dd076511208 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:12.648314 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc3f77-b5e6-4e72-9cb9-7dd076511208 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:12.648786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc3f77-b5e6-4e72-9cb9-7dd076511208 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:12.649770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc3f77-b5e6-4e72-9cb9-7dd076511208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:12.650210 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc3f77-b5e6-4e72-9cb9-7dd076511208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:12.652276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fc3f77-b5e6-4e72-9cb9-7dd076511208 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:12.668568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 609/1243] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:40:12.675629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc04f18-57fc-49f7-bf73-3c91e38f0a40 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:12.774519 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:12.810719 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc04f18-57fc-49f7-bf73-3c91e38f0a40 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:12.864915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558412e7-bdc6-4e0c-9d4b-9caf4e09413b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:12.865559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558412e7-bdc6-4e0c-9d4b-9caf4e09413b None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:12.865831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558412e7-bdc6-4e0c-9d4b-9caf4e09413b None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:12.867266 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558412e7-bdc6-4e0c-9d4b-9caf4e09413b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:12.874072 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558412e7-bdc6-4e0c-9d4b-9caf4e09413b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:12.876513 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558412e7-bdc6-4e0c-9d4b-9caf4e09413b None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-558412e7-bdc6-4e0c-9d4b-9caf4e09413b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:40:12.897799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:12.899193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:12.900434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:40:12.900434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:40:12.900434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:40:12.900434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:40:12.900434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:40:12.900434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:40:12.900434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:40:12.900434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:12.900434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 635/1244] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:40:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:12.911495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc3f77-b5e6-4e72-9cb9-7dd076511208 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:13.047995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc3f77-b5e6-4e72-9cb9-7dd076511208 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:13.095483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347276f-d3d6-451c-bb41-f1d412b82c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:13.096106 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347276f-d3d6-451c-bb41-f1d412b82c60 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:13.096383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347276f-d3d6-451c-bb41-f1d412b82c60 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:13.096658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347276f-d3d6-451c-bb41-f1d412b82c60 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:13.097510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347276f-d3d6-451c-bb41-f1d412b82c60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:13.097801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347276f-d3d6-451c-bb41-f1d412b82c60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:13.099276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347276f-d3d6-451c-bb41-f1d412b82c60 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:13.115714 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 610/1245] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:40:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:13.122506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558412e7-bdc6-4e0c-9d4b-9caf4e09413b None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:13.255790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558412e7-bdc6-4e0c-9d4b-9caf4e09413b None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:13.301681 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a4b45-421a-4d73-910a-ba5fdff0a130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:13.302305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a4b45-421a-4d73-910a-ba5fdff0a130 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:13.302609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a4b45-421a-4d73-910a-ba5fdff0a130 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:13.303027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a4b45-421a-4d73-910a-ba5fdff0a130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:13.304253 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a4b45-421a-4d73-910a-ba5fdff0a130 None admin] RBAC: Adding query filter params (role.id=02001b18565340ce984343c5d36daaf9, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=cce71f93c80d433b830c448d0a7a14e3) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:13.304548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a4b45-421a-4d73-910a-ba5fdff0a130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:13.306003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a4b45-421a-4d73-910a-ba5fdff0a130 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:13.312852 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 636/1246] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:40:13 2022] GET /identity/v3/role_assignments?user.id=cce71f93c80d433b830c448d0a7a14e3&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=02001b18565340ce984343c5d36daaf9 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:40:15.588058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf50d3-b621-48e4-a604-20156fb3c2d4 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:15.588729 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf50d3-b621-48e4-a604-20156fb3c2d4 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:15.588729 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf50d3-b621-48e4-a604-20156fb3c2d4 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:15.589938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 611/1247] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:40:15.596100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900574d-b18b-4db0-a9e6-a284f5564136 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:15.596599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900574d-b18b-4db0-a9e6-a284f5564136 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:15.596694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900574d-b18b-4db0-a9e6-a284f5564136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:15.603466 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:15.618910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-1900574d-b18b-4db0-a9e6-a284f5564136 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:40:15.620684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1900574d-b18b-4db0-a9e6-a284f5564136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:40:15.707229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 637/1248] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:40:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:40:15.731759 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf50d3-b621-48e4-a604-20156fb3c2d4 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:15.754989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:15.889554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf50d3-b621-48e4-a604-20156fb3c2d4 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:15.938096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db273fd6-426b-4bda-be79-4bbbdb375c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:15.938786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db273fd6-426b-4bda-be79-4bbbdb375c4e None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:15.939056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db273fd6-426b-4bda-be79-4bbbdb375c4e None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:15.939324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db273fd6-426b-4bda-be79-4bbbdb375c4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:15.952190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db273fd6-426b-4bda-be79-4bbbdb375c4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-db273fd6-426b-4bda-be79-4bbbdb375c4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:40:15.959010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:15.960950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:15.962547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:15.962547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:15.962547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:15.962547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:15.962547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:40:15.962547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:40:15.962547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:40:15.962547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:40:15.962547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:40:15.962547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:40:15.962547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:40:15.962547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:15.962547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 612/1249] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:40:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:40:15.967792 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900574d-b18b-4db0-a9e6-a284f5564136 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:16.084396 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900574d-b18b-4db0-a9e6-a284f5564136 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:16.124152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:16.131496 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1428b5d-78f0-44d7-af62-51e3b8e626e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:16.132538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1428b5d-78f0-44d7-af62-51e3b8e626e8 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:16.133124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1428b5d-78f0-44d7-af62-51e3b8e626e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:16.133677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1428b5d-78f0-44d7-af62-51e3b8e626e8 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:16.134750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1428b5d-78f0-44d7-af62-51e3b8e626e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:16.135225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1428b5d-78f0-44d7-af62-51e3b8e626e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:16.137171 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1428b5d-78f0-44d7-af62-51e3b8e626e8 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:16.149418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 638/1250] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:40:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:16.156040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db273fd6-426b-4bda-be79-4bbbdb375c4e None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:16.260895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:16.291320 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db273fd6-426b-4bda-be79-4bbbdb375c4e None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:16.334366 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9dcc3-da14-41c8-b8d1-94ce957df723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:16.335343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9dcc3-da14-41c8-b8d1-94ce957df723 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:16.335488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9dcc3-da14-41c8-b8d1-94ce957df723 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:16.335809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9dcc3-da14-41c8-b8d1-94ce957df723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:16.348949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9dcc3-da14-41c8-b8d1-94ce957df723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:16.350655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9dcc3-da14-41c8-b8d1-94ce957df723 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-8dc9dcc3-da14-41c8-b8d1-94ce957df723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:40:16.376165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:16.377200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:16.378123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:40:16.378123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:40:16.378123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:16.378123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:16.378721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 613/1251] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:40:16.384458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1428b5d-78f0-44d7-af62-51e3b8e626e8 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:16.534910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1428b5d-78f0-44d7-af62-51e3b8e626e8 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:16.588707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f39417-17c7-49f6-8a42-b620a925433c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:16.589484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f39417-17c7-49f6-8a42-b620a925433c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:16.589777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f39417-17c7-49f6-8a42-b620a925433c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:16.590060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f39417-17c7-49f6-8a42-b620a925433c None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:16.590820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f39417-17c7-49f6-8a42-b620a925433c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:16.591132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f39417-17c7-49f6-8a42-b620a925433c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:16.592728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f39417-17c7-49f6-8a42-b620a925433c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:16.610372 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 639/1252] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:16.628294 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9dcc3-da14-41c8-b8d1-94ce957df723 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:16.765937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9dcc3-da14-41c8-b8d1-94ce957df723 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:16.804988 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:16.817332 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a48e1e-1cb4-467d-8f2f-1cb7aec57b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:16.818158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a48e1e-1cb4-467d-8f2f-1cb7aec57b0b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:16.818488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a48e1e-1cb4-467d-8f2f-1cb7aec57b0b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:16.818804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a48e1e-1cb4-467d-8f2f-1cb7aec57b0b None admin] PATH_INFO: `/v3/users/placement` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:16.828923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a48e1e-1cb4-467d-8f2f-1cb7aec57b0b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:16.830861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a48e1e-1cb4-467d-8f2f-1cb7aec57b0b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-00a48e1e-1cb4-467d-8f2f-1cb7aec57b0b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:40:16.846732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:16.854559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:16.856196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:16.856196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:16.856196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:40:16.856196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:40:16.856196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:40:16.856196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:40:16.856196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:40:16.856196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:40:16.856196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:40:16.856196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:16.856196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 614/1253] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:16.857224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f39417-17c7-49f6-8a42-b620a925433c None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:16.941309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:16.996378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f39417-17c7-49f6-8a42-b620a925433c None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:17.047757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad83bf-0b7b-4422-9921-edd9747087e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:17.048552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad83bf-0b7b-4422-9921-edd9747087e0 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:17.048929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad83bf-0b7b-4422-9921-edd9747087e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:17.049323 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad83bf-0b7b-4422-9921-edd9747087e0 None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:17.051473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad83bf-0b7b-4422-9921-edd9747087e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:17.052038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad83bf-0b7b-4422-9921-edd9747087e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:17.057302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad83bf-0b7b-4422-9921-edd9747087e0 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:17.089103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 640/1254] 10.222.0.86 () {56 vars in 1143 bytes} [Fri Aug 5 19:40:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:17.096697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a48e1e-1cb4-467d-8f2f-1cb7aec57b0b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:17.238497 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a48e1e-1cb4-467d-8f2f-1cb7aec57b0b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:17.285500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a9d10-387e-4b08-a214-1ba8b665f05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:17.286274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a9d10-387e-4b08-a214-1ba8b665f05b None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:17.286681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a9d10-387e-4b08-a214-1ba8b665f05b None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:17.287210 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a9d10-387e-4b08-a214-1ba8b665f05b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:17.362368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:17.376439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a9d10-387e-4b08-a214-1ba8b665f05b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:17.378024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a9d10-387e-4b08-a214-1ba8b665f05b None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:17.502843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 615/1255] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:40:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:40:17.508887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad83bf-0b7b-4422-9921-edd9747087e0 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:17.652953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad83bf-0b7b-4422-9921-edd9747087e0 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:17.667839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:17.692632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc126b47-9a15-4a6d-8436-01ee07aff05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:17.693278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc126b47-9a15-4a6d-8436-01ee07aff05f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:17.693554 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc126b47-9a15-4a6d-8436-01ee07aff05f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:17.693820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc126b47-9a15-4a6d-8436-01ee07aff05f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:17.706080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc126b47-9a15-4a6d-8436-01ee07aff05f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:17.707764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc126b47-9a15-4a6d-8436-01ee07aff05f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-fc126b47-9a15-4a6d-8436-01ee07aff05f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:40:17.724359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:17.725541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:17.726451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:40:17.726451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:40:17.726451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:17.726451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:17.726451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 641/1256] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:17.731995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a9d10-387e-4b08-a214-1ba8b665f05b None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:17.850181 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:17.865891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a9d10-387e-4b08-a214-1ba8b665f05b None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:17.910647 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac2da2-d71b-4699-898a-c763d0db0895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:17.911296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac2da2-d71b-4699-898a-c763d0db0895 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:17.911619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac2da2-d71b-4699-898a-c763d0db0895 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:17.911898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac2da2-d71b-4699-898a-c763d0db0895 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:17.912556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac2da2-d71b-4699-898a-c763d0db0895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:17.912818 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac2da2-d71b-4699-898a-c763d0db0895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:17.914197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac2da2-d71b-4699-898a-c763d0db0895 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:17.930676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 616/1257] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:17.934652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc126b47-9a15-4a6d-8436-01ee07aff05f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:18.055396 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc126b47-9a15-4a6d-8436-01ee07aff05f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:18.091290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f84c8e-49e6-4008-8ca2-8de6333f9a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:18.091922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f84c8e-49e6-4008-8ca2-8de6333f9a7c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:18.092191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f84c8e-49e6-4008-8ca2-8de6333f9a7c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:18.092456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f84c8e-49e6-4008-8ca2-8de6333f9a7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:18.100278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f84c8e-49e6-4008-8ca2-8de6333f9a7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:18.101668 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f84c8e-49e6-4008-8ca2-8de6333f9a7c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-47f84c8e-49e6-4008-8ca2-8de6333f9a7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:40:18.115229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:18.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:18.117140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:40:18.117140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:40:18.117140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:40:18.117140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:40:18.117140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:40:18.117140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:40:18.117140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:40:18.117140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:18.117140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 642/1258] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:40:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:18.122688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac2da2-d71b-4699-898a-c763d0db0895 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:18.239340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac2da2-d71b-4699-898a-c763d0db0895 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:18.278666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ac169-0f69-4bee-af74-a189c3fcb939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:18.279318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ac169-0f69-4bee-af74-a189c3fcb939 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:18.279594 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ac169-0f69-4bee-af74-a189c3fcb939 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:18.279895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ac169-0f69-4bee-af74-a189c3fcb939 None admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:18.281210 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ac169-0f69-4bee-af74-a189c3fcb939 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:18.281592 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ac169-0f69-4bee-af74-a189c3fcb939 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:18.283273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ac169-0f69-4bee-af74-a189c3fcb939 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:18.297148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 617/1259] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:40:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:18.303771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f84c8e-49e6-4008-8ca2-8de6333f9a7c None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:18.334710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:18.429977 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f84c8e-49e6-4008-8ca2-8de6333f9a7c None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:18.486345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090867f-a9ec-45e7-9cb2-1652a5a17a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:18.487033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090867f-a9ec-45e7-9cb2-1652a5a17a6e None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:18.487283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090867f-a9ec-45e7-9cb2-1652a5a17a6e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:18.487574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090867f-a9ec-45e7-9cb2-1652a5a17a6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:18.488380 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090867f-a9ec-45e7-9cb2-1652a5a17a6e None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=cce71f93c80d433b830c448d0a7a14e3) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:18.488705 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090867f-a9ec-45e7-9cb2-1652a5a17a6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:18.490235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090867f-a9ec-45e7-9cb2-1652a5a17a6e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:18.505182 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 643/1260] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:40:18 2022] GET /identity/v3/role_assignments?user.id=cce71f93c80d433b830c448d0a7a14e3&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=93eb204498fc49d297bcd1841e00818d => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:40:20.707405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b0567-2c7d-4ea6-9098-f58b1f099bef None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:20.707955 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b0567-2c7d-4ea6-9098-f58b1f099bef None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:20.707955 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b0567-2c7d-4ea6-9098-f58b1f099bef None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:20.709335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 618/1261] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:40:20.715427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b1a5b-5b4e-4e71-a5b2-b7c755e7bd3f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:20.715957 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b1a5b-5b4e-4e71-a5b2-b7c755e7bd3f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:20.715957 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b1a5b-5b4e-4e71-a5b2-b7c755e7bd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:20.737225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-a71b1a5b-5b4e-4e71-a5b2-b7c755e7bd3f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:40:20.738384 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a71b1a5b-5b4e-4e71-a5b2-b7c755e7bd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:40:20.815257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 644/1262] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:40:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:40:20.838460 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b0567-2c7d-4ea6-9098-f58b1f099bef None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:20.851256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:20.963558 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b0567-2c7d-4ea6-9098-f58b1f099bef None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:21.000265 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1682482-0323-40e2-a70b-1171870b2ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:21.001237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1682482-0323-40e2-a70b-1171870b2ae8 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:21.001521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1682482-0323-40e2-a70b-1171870b2ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:21.001791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1682482-0323-40e2-a70b-1171870b2ae8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:21.016269 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1682482-0323-40e2-a70b-1171870b2ae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-b1682482-0323-40e2-a70b-1171870b2ae8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:40:21.021562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:21.022546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:21.023451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:21.023451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:21.023451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:21.023451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:21.023451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:40:21.023451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:40:21.023451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:40:21.023451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:40:21.023451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:40:21.023451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:40:21.023451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:40:21.023451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:21.023451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 619/1263] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:40:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:21.029190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b1a5b-5b4e-4e71-a5b2-b7c755e7bd3f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:21.052221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:21.167302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b1a5b-5b4e-4e71-a5b2-b7c755e7bd3f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:21.202139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ad833-7ce1-47fd-b878-156c5a491b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:21.203505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ad833-7ce1-47fd-b878-156c5a491b08 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:21.203802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ad833-7ce1-47fd-b878-156c5a491b08 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:21.204082 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ad833-7ce1-47fd-b878-156c5a491b08 None admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:21.204792 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ad833-7ce1-47fd-b878-156c5a491b08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:21.205095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ad833-7ce1-47fd-b878-156c5a491b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:21.206465 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ad833-7ce1-47fd-b878-156c5a491b08 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:21.214636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 645/1264] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:40:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:21.222379 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1682482-0323-40e2-a70b-1171870b2ae8 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:21.300271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:21.336761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1682482-0323-40e2-a70b-1171870b2ae8 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:21.370887 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab3ce7-d379-473c-96fc-9f1497f4c0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:21.371515 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab3ce7-d379-473c-96fc-9f1497f4c0c1 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:21.371793 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab3ce7-d379-473c-96fc-9f1497f4c0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:21.372065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab3ce7-d379-473c-96fc-9f1497f4c0c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:21.384255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab3ce7-d379-473c-96fc-9f1497f4c0c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:21.385704 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab3ce7-d379-473c-96fc-9f1497f4c0c1 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-8dab3ce7-d379-473c-96fc-9f1497f4c0c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:40:21.402225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:21.403429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:21.404353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:40:21.404353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:40:21.404353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:21.404353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:21.404353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 620/1265] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:21.409350 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ad833-7ce1-47fd-b878-156c5a491b08 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:21.491679 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:21.512807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ad833-7ce1-47fd-b878-156c5a491b08 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:21.547660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf1b0f-3bff-4a1f-85dc-7bd0808d2fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:21.548383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf1b0f-3bff-4a1f-85dc-7bd0808d2fd1 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:21.548550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf1b0f-3bff-4a1f-85dc-7bd0808d2fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:21.548870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf1b0f-3bff-4a1f-85dc-7bd0808d2fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:21.549629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf1b0f-3bff-4a1f-85dc-7bd0808d2fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:21.549932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf1b0f-3bff-4a1f-85dc-7bd0808d2fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:21.551505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf1b0f-3bff-4a1f-85dc-7bd0808d2fd1 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:21.564749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 646/1266] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:21.570981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab3ce7-d379-473c-96fc-9f1497f4c0c1 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:21.671165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab3ce7-d379-473c-96fc-9f1497f4c0c1 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:21.691729 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:21.706138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4dd8d2-4590-4579-aea4-b6a516579c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:21.706822 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dd8d2-4590-4579-aea4-b6a516579c6f None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:21.707083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dd8d2-4590-4579-aea4-b6a516579c6f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:21.707350 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dd8d2-4590-4579-aea4-b6a516579c6f None admin] PATH_INFO: `/v3/users/placement` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:21.714834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4dd8d2-4590-4579-aea4-b6a516579c6f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:21.716261 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4dd8d2-4590-4579-aea4-b6a516579c6f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-1d4dd8d2-4590-4579-aea4-b6a516579c6f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:40:21.727420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:21.728534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:21.729786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:21.729786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:21.729786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:40:21.729786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:40:21.729786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:40:21.729786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:40:21.729786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:40:21.729786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:40:21.729786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:40:21.729786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:21.729786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 621/1267] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:21.734481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf1b0f-3bff-4a1f-85dc-7bd0808d2fd1 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:21.833131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf1b0f-3bff-4a1f-85dc-7bd0808d2fd1 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:21.866918 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d9577-abbf-4a60-b15a-37f7d26098ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:21.867529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d9577-abbf-4a60-b15a-37f7d26098ab None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:21.867811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d9577-abbf-4a60-b15a-37f7d26098ab None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:21.868080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d9577-abbf-4a60-b15a-37f7d26098ab None admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:21.868859 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d9577-abbf-4a60-b15a-37f7d26098ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:21.869162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d9577-abbf-4a60-b15a-37f7d26098ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:21.870578 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d9577-abbf-4a60-b15a-37f7d26098ab None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:21.881498 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:21.893552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 647/1268] 10.222.0.86 () {56 vars in 1143 bytes} [Fri Aug 5 19:40:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:21.900023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4dd8d2-4590-4579-aea4-b6a516579c6f None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:22.009206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4dd8d2-4590-4579-aea4-b6a516579c6f None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:22.042026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554885b0-8e52-4577-a731-7436be01c3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:22.042679 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554885b0-8e52-4577-a731-7436be01c3ff None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:22.042961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554885b0-8e52-4577-a731-7436be01c3ff None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:22.043230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554885b0-8e52-4577-a731-7436be01c3ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:22.053764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554885b0-8e52-4577-a731-7436be01c3ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:22.055232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554885b0-8e52-4577-a731-7436be01c3ff None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-554885b0-8e52-4577-a731-7436be01c3ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:40:22.069794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:22.070984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:22.072083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:40:22.072083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:40:22.072083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:22.072083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:22.072083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 622/1269] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:22.076932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d9577-abbf-4a60-b15a-37f7d26098ab None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:22.175268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d9577-abbf-4a60-b15a-37f7d26098ab None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:22.210419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a921784-d940-40ce-90f3-743604796631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:22.211063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a921784-d940-40ce-90f3-743604796631 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:22.211339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a921784-d940-40ce-90f3-743604796631 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:22.211611 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a921784-d940-40ce-90f3-743604796631 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:22.212298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a921784-d940-40ce-90f3-743604796631 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:22.212598 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a921784-d940-40ce-90f3-743604796631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:22.213962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a921784-d940-40ce-90f3-743604796631 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:22.225193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 648/1270] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:22.230943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554885b0-8e52-4577-a731-7436be01c3ff None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:22.262954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:22.334902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554885b0-8e52-4577-a731-7436be01c3ff None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:22.367523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fe24d-9422-4ba3-bdc3-77064c84aabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:22.368152 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fe24d-9422-4ba3-bdc3-77064c84aabe None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:22.368429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fe24d-9422-4ba3-bdc3-77064c84aabe None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:22.368700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fe24d-9422-4ba3-bdc3-77064c84aabe None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:22.375175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fe24d-9422-4ba3-bdc3-77064c84aabe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:22.377181 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fe24d-9422-4ba3-bdc3-77064c84aabe None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-cf4fe24d-9422-4ba3-bdc3-77064c84aabe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:40:22.388344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:22.389404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:22.390712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:40:22.390712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:40:22.390712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:40:22.390712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:40:22.390712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:40:22.390712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:40:22.390712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:40:22.390712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:22.390712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 623/1271] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:40:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:40:22.396090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a921784-d940-40ce-90f3-743604796631 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:22.457223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:22.499747 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a921784-d940-40ce-90f3-743604796631 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:22.534147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28366ee7-a141-42d1-8af5-661006387e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:22.534791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28366ee7-a141-42d1-8af5-661006387e1e None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:22.535067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28366ee7-a141-42d1-8af5-661006387e1e None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:22.535346 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28366ee7-a141-42d1-8af5-661006387e1e None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:22.536097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28366ee7-a141-42d1-8af5-661006387e1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:22.536399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28366ee7-a141-42d1-8af5-661006387e1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:22.537833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28366ee7-a141-42d1-8af5-661006387e1e None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:22.550258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 649/1272] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:40:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:22.556407 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fe24d-9422-4ba3-bdc3-77064c84aabe None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:22.652034 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:22.663098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fe24d-9422-4ba3-bdc3-77064c84aabe None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:22.696473 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc54753-7033-4f33-acfe-291ccb889a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:22.697128 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc54753-7033-4f33-acfe-291ccb889a5f None admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:22.697403 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc54753-7033-4f33-acfe-291ccb889a5f None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:22.697668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc54753-7033-4f33-acfe-291ccb889a5f None admin] PATH_INFO: `/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/cce71f93c80d433b830c448d0a7a14e3/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:22.699620 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc54753-7033-4f33-acfe-291ccb889a5f None admin] RBAC: Authorizing `identity:create_grant(project_id=9c73d2cfaa1c47829ca6936186d2bb26, user_id=cce71f93c80d433b830c448d0a7a14e3, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:22.701044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc54753-7033-4f33-acfe-291ccb889a5f None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:22.718611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 624/1273] 10.222.0.86 () {58 vars in 1439 bytes} [Fri Aug 5 19:40:22 2022] PUT /identity/v3/projects/9c73d2cfaa1c47829ca6936186d2bb26/users/cce71f93c80d433b830c448d0a7a14e3/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:40:24.842035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45015724-bdfd-4ea6-ad93-de3d189d2ee0 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:24.842615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45015724-bdfd-4ea6-ad93-de3d189d2ee0 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:24.842615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45015724-bdfd-4ea6-ad93-de3d189d2ee0 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:24.843963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 650/1274] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:40:24.850048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c127982-c5bc-4f20-a603-13b5ea3a21fd None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:24.850537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c127982-c5bc-4f20-a603-13b5ea3a21fd None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:24.850670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c127982-c5bc-4f20-a603-13b5ea3a21fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:24.869011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-6c127982-c5bc-4f20-a603-13b5ea3a21fd None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:40:24.870191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6c127982-c5bc-4f20-a603-13b5ea3a21fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:40:24.946010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 625/1275] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:40:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:40:24.972675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45015724-bdfd-4ea6-ad93-de3d189d2ee0 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:25.080654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45015724-bdfd-4ea6-ad93-de3d189d2ee0 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:25.100997 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:25.118832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34802725-5a28-48ef-a583-7a707b2f167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:25.119457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34802725-5a28-48ef-a583-7a707b2f167f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:25.119729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34802725-5a28-48ef-a583-7a707b2f167f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:25.119996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34802725-5a28-48ef-a583-7a707b2f167f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:25.130333 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34802725-5a28-48ef-a583-7a707b2f167f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-34802725-5a28-48ef-a583-7a707b2f167f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:40:25.135141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:25.136287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:25.137353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:25.137353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:25.137353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:25.137353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:25.137353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:40:25.137353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:40:25.137353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:40:25.137353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:40:25.137353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:40:25.137353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:40:25.137353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:40:25.137353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:25.137353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 651/1276] 10.222.0.86 () {56 vars in 1096 bytes} [Fri Aug 5 19:40:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:25.142608 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c127982-c5bc-4f20-a603-13b5ea3a21fd None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:25.226687 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:25.246825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c127982-c5bc-4f20-a603-13b5ea3a21fd None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:25.282256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04cf7b-3f08-4aea-a138-17e55706f336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:25.282925 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04cf7b-3f08-4aea-a138-17e55706f336 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:25.283201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04cf7b-3f08-4aea-a138-17e55706f336 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:25.283468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04cf7b-3f08-4aea-a138-17e55706f336 None admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:25.284170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04cf7b-3f08-4aea-a138-17e55706f336 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:25.284470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04cf7b-3f08-4aea-a138-17e55706f336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:25.285833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04cf7b-3f08-4aea-a138-17e55706f336 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:25.294027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 626/1277] 10.222.0.86 () {56 vars in 1099 bytes} [Fri Aug 5 19:40:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:40:25.300055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34802725-5a28-48ef-a583-7a707b2f167f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:25.416845 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34802725-5a28-48ef-a583-7a707b2f167f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:25.452633 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca669105-6d25-4c5a-ab0c-f359cfa10a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:25.453257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca669105-6d25-4c5a-ab0c-f359cfa10a3c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:25.453825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca669105-6d25-4c5a-ab0c-f359cfa10a3c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:25.454466 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca669105-6d25-4c5a-ab0c-f359cfa10a3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:25.466151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca669105-6d25-4c5a-ab0c-f359cfa10a3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:25.467677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca669105-6d25-4c5a-ab0c-f359cfa10a3c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-ca669105-6d25-4c5a-ab0c-f359cfa10a3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:40:25.482276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:25.483427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:25.484633 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:40:25.484633 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:40:25.484633 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:25.484633 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:25.484633 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 652/1278] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:25.490645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04cf7b-3f08-4aea-a138-17e55706f336 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:25.591239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04cf7b-3f08-4aea-a138-17e55706f336 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:25.624919 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb6acf-a48c-496b-8f8f-cff52331a519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:25.626258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6acf-a48c-496b-8f8f-cff52331a519 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:25.626838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6acf-a48c-496b-8f8f-cff52331a519 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:25.626941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6acf-a48c-496b-8f8f-cff52331a519 None admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:25.627858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb6acf-a48c-496b-8f8f-cff52331a519 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:25.628165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb6acf-a48c-496b-8f8f-cff52331a519 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:25.629555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb6acf-a48c-496b-8f8f-cff52331a519 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:25.642697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 627/1279] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:25.648536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca669105-6d25-4c5a-ab0c-f359cfa10a3c None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:25.680955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:25.754001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca669105-6d25-4c5a-ab0c-f359cfa10a3c None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:25.786678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bfbc2-e0a0-4d8e-b5f0-e2feb19f0692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:25.787303 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bfbc2-e0a0-4d8e-b5f0-e2feb19f0692 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:25.787581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bfbc2-e0a0-4d8e-b5f0-e2feb19f0692 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:25.787850 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bfbc2-e0a0-4d8e-b5f0-e2feb19f0692 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:25.795101 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bfbc2-e0a0-4d8e-b5f0-e2feb19f0692 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:25.796557 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bfbc2-e0a0-4d8e-b5f0-e2feb19f0692 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-009bfbc2-e0a0-4d8e-b5f0-e2feb19f0692 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:40:25.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:25.809565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:25.810784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:25.810784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:25.810784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:40:25.810784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:40:25.810784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:40:25.810784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:40:25.810784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:40:25.810784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:40:25.810784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 05 19:40:25.810784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:25.810784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 653/1280] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:25.816203 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb6acf-a48c-496b-8f8f-cff52331a519 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:25.820038 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:25.929815 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb6acf-a48c-496b-8f8f-cff52331a519 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:25.963526 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf43c8-1121-4f4c-b49d-bdb87e462e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:25.964052 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf43c8-1121-4f4c-b49d-bdb87e462e4c None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:25.964362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf43c8-1121-4f4c-b49d-bdb87e462e4c None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:25.964636 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faf43c8-1121-4f4c-b49d-bdb87e462e4c None admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:25.965497 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf43c8-1121-4f4c-b49d-bdb87e462e4c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:25.965796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf43c8-1121-4f4c-b49d-bdb87e462e4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:25.967269 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faf43c8-1121-4f4c-b49d-bdb87e462e4c None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:25.986553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 628/1281] 10.222.0.86 () {56 vars in 1143 bytes} [Fri Aug 5 19:40:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:25.992515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bfbc2-e0a0-4d8e-b5f0-e2feb19f0692 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:26.084294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:26.094834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bfbc2-e0a0-4d8e-b5f0-e2feb19f0692 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:26.129413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b56da-013a-49a0-b93f-b6efc47cf399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:26.130037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b56da-013a-49a0-b93f-b6efc47cf399 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:26.130315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b56da-013a-49a0-b93f-b6efc47cf399 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:26.130586 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b56da-013a-49a0-b93f-b6efc47cf399 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:26.203567 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b56da-013a-49a0-b93f-b6efc47cf399 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:26.205075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2b56da-013a-49a0-b93f-b6efc47cf399 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:26.293541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:26.304744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 654/1282] 10.222.0.86 () {58 vars in 1282 bytes} [Fri Aug 5 19:40:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:40:26.310656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf43c8-1121-4f4c-b49d-bdb87e462e4c None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:26.385565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:26.427979 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faf43c8-1121-4f4c-b49d-bdb87e462e4c None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:26.463019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8e11f-ba49-4a9e-8282-2e75772b0fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:26.463659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8e11f-ba49-4a9e-8282-2e75772b0fb5 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:26.463943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8e11f-ba49-4a9e-8282-2e75772b0fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:26.464211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8e11f-ba49-4a9e-8282-2e75772b0fb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:26.474340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8e11f-ba49-4a9e-8282-2e75772b0fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:26.475844 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8e11f-ba49-4a9e-8282-2e75772b0fb5 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-cdc8e11f-ba49-4a9e-8282-2e75772b0fb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:40:26.491157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:26.492114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:26.493027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:40:26.493027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:40:26.493027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:40:26.493027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:26.493027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 629/1283] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:40:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:26.499362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b56da-013a-49a0-b93f-b6efc47cf399 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:26.611030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b56da-013a-49a0-b93f-b6efc47cf399 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:26.658603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731aae7f-eb38-4e9d-9d0c-ec0d4a36bd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:26.659365 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731aae7f-eb38-4e9d-9d0c-ec0d4a36bd32 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:26.659694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731aae7f-eb38-4e9d-9d0c-ec0d4a36bd32 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:26.659968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731aae7f-eb38-4e9d-9d0c-ec0d4a36bd32 None admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:26.660720 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731aae7f-eb38-4e9d-9d0c-ec0d4a36bd32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:26.661025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731aae7f-eb38-4e9d-9d0c-ec0d4a36bd32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:26.662881 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731aae7f-eb38-4e9d-9d0c-ec0d4a36bd32 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:26.677797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 655/1284] 10.222.0.86 () {56 vars in 1109 bytes} [Fri Aug 5 19:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:26.683448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8e11f-ba49-4a9e-8282-2e75772b0fb5 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:26.788904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8e11f-ba49-4a9e-8282-2e75772b0fb5 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:26.803012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:26.823921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be851855-fbf6-4486-a179-fc9326252cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:26.824554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be851855-fbf6-4486-a179-fc9326252cbf None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:26.824833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be851855-fbf6-4486-a179-fc9326252cbf None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:26.825102 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be851855-fbf6-4486-a179-fc9326252cbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:26.834849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be851855-fbf6-4486-a179-fc9326252cbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:26.836650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be851855-fbf6-4486-a179-fc9326252cbf None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application [None req-be851855-fbf6-4486-a179-fc9326252cbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:40:26.852921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:26.854090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:26.855908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:40:26.855908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:40:26.855908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:40:26.855908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:40:26.855908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:40:26.855908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:40:26.855908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:40:26.855908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: ERROR keystone.server.flask.application Aug 05 19:40:26.855908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 630/1285] 10.222.0.86 () {56 vars in 1111 bytes} [Fri Aug 5 19:40:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:26.860750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731aae7f-eb38-4e9d-9d0c-ec0d4a36bd32 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:26.929479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:26.982684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731aae7f-eb38-4e9d-9d0c-ec0d4a36bd32 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:27.017762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd4981-5d80-429f-84d0-44f370505527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:27.018602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd4981-5d80-429f-84d0-44f370505527 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:27.018602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd4981-5d80-429f-84d0-44f370505527 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:27.018974 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd4981-5d80-429f-84d0-44f370505527 None admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:27.019627 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd4981-5d80-429f-84d0-44f370505527 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:27.019926 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd4981-5d80-429f-84d0-44f370505527 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:27.021331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd4981-5d80-429f-84d0-44f370505527 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:27.033506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 656/1286] 10.222.0.86 () {56 vars in 1148 bytes} [Fri Aug 5 19:40:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:40:27.039547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be851855-fbf6-4486-a179-fc9326252cbf None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:27.143716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be851855-fbf6-4486-a179-fc9326252cbf None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:27.177526 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa29ec-72c5-4908-878c-c8b0ed673437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:27.178206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa29ec-72c5-4908-878c-c8b0ed673437 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:27.178484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa29ec-72c5-4908-878c-c8b0ed673437 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:27.178784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa29ec-72c5-4908-878c-c8b0ed673437 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:27.179620 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa29ec-72c5-4908-878c-c8b0ed673437 None admin] RBAC: Adding query filter params (role.id=93eb204498fc49d297bcd1841e00818d, scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26, user.id=cce71f93c80d433b830c448d0a7a14e3) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:27.179867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa29ec-72c5-4908-878c-c8b0ed673437 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:27.181283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa29ec-72c5-4908-878c-c8b0ed673437 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:27.187231 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 631/1287] 10.222.0.86 () {56 vars in 1374 bytes} [Fri Aug 5 19:40:27 2022] GET /identity/v3/role_assignments?user.id=cce71f93c80d433b830c448d0a7a14e3&scope.project.id=9c73d2cfaa1c47829ca6936186d2bb26&role.id=93eb204498fc49d297bcd1841e00818d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:29.229588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bb744-45f3-4ae5-8f26-74dfc26ec253 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:29.230169 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bb744-45f3-4ae5-8f26-74dfc26ec253 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:29.230169 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bb744-45f3-4ae5-8f26-74dfc26ec253 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:29.231600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 657/1288] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:40:29.237483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e48826-2b9f-464a-9164-cb6c1a890d60 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:29.237971 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e48826-2b9f-464a-9164-cb6c1a890d60 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:29.238082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e48826-2b9f-464a-9164-cb6c1a890d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:29.257899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-03e48826-2b9f-464a-9164-cb6c1a890d60 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:40:29.259137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-03e48826-2b9f-464a-9164-cb6c1a890d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:40:29.291414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:29.332088 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 632/1289] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:40:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:40:29.356842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bb744-45f3-4ae5-8f26-74dfc26ec253 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:29.471459 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bb744-45f3-4ae5-8f26-74dfc26ec253 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:29.483223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:29.508846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6848853-3331-4dca-8dfa-fbf495084d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:29.509481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6848853-3331-4dca-8dfa-fbf495084d9f None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:29.509754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6848853-3331-4dca-8dfa-fbf495084d9f None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:29.510021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6848853-3331-4dca-8dfa-fbf495084d9f None admin] PATH_INFO: `/v3/services/placement` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:29.510669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6848853-3331-4dca-8dfa-fbf495084d9f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:29.512023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6848853-3331-4dca-8dfa-fbf495084d9f None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-a6848853-3331-4dca-8dfa-fbf495084d9f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:29.521674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:29.523020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:29.523958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 658/1290] 10.222.0.86 () {56 vars in 1117 bytes} [Fri Aug 5 19:40:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:29.528878 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e48826-2b9f-464a-9164-cb6c1a890d60 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:29.641777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e48826-2b9f-464a-9164-cb6c1a890d60 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:29.676027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ef430-685e-418e-bd75-29b80aff18a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:29.676635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ef430-685e-418e-bd75-29b80aff18a0 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:29.676908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ef430-685e-418e-bd75-29b80aff18a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:29.677175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ef430-685e-418e-bd75-29b80aff18a0 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:29.677846 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ef430-685e-418e-bd75-29b80aff18a0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:29.678142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ef430-685e-418e-bd75-29b80aff18a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:29.679643 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ef430-685e-418e-bd75-29b80aff18a0 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:29.684686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 633/1291] 10.222.0.86 () {56 vars in 1116 bytes} [Fri Aug 5 19:40:29 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:29.691607 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6848853-3331-4dca-8dfa-fbf495084d9f None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:29.793235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6848853-3331-4dca-8dfa-fbf495084d9f None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:29.827112 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe79f6-fa81-4d5a-a6fb-c5bfd0cdd454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:29.827722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe79f6-fa81-4d5a-a6fb-c5bfd0cdd454 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:29.827992 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe79f6-fa81-4d5a-a6fb-c5bfd0cdd454 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:29.828296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe79f6-fa81-4d5a-a6fb-c5bfd0cdd454 None admin] PATH_INFO: `/v3/services` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:29.829699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe79f6-fa81-4d5a-a6fb-c5bfd0cdd454 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:29.830008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe79f6-fa81-4d5a-a6fb-c5bfd0cdd454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:29.831412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe79f6-fa81-4d5a-a6fb-c5bfd0cdd454 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:29.836169 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 659/1292] 10.222.0.86 () {56 vars in 1116 bytes} [Fri Aug 5 19:40:29 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:40:31.896952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb532-b69c-4ff9-bd10-66c4e069d7c3 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:31.897494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb532-b69c-4ff9-bd10-66c4e069d7c3 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:31.897494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb532-b69c-4ff9-bd10-66c4e069d7c3 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:31.898871 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 634/1293] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:40:31.905336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f022e92-468f-4edb-880e-9fee3d9bc467 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:31.905760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f022e92-468f-4edb-880e-9fee3d9bc467 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:31.905894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f022e92-468f-4edb-880e-9fee3d9bc467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:31.926290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-2f022e92-468f-4edb-880e-9fee3d9bc467 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:40:31.927896 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2f022e92-468f-4edb-880e-9fee3d9bc467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:40:31.961576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:31.997981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 660/1294] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:40:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:40:32.021783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacb532-b69c-4ff9-bd10-66c4e069d7c3 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:32.052154 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:32.121411 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacb532-b69c-4ff9-bd10-66c4e069d7c3 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:32.156374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4bea4-a557-4f97-a825-7788a19fec75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:32.157444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4bea4-a557-4f97-a825-7788a19fec75 None admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:32.157718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4bea4-a557-4f97-a825-7788a19fec75 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:32.157985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4bea4-a557-4f97-a825-7788a19fec75 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:32.158784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4bea4-a557-4f97-a825-7788a19fec75 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:32.160034 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4bea4-a557-4f97-a825-7788a19fec75 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:32.175142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 635/1295] 10.222.0.86 () {60 vars in 1141 bytes} [Fri Aug 5 19:40:32 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:40:34.210309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62bcb0-2710-4b54-85ef-1876fd51e106 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:34.210878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62bcb0-2710-4b54-85ef-1876fd51e106 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:34.210878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62bcb0-2710-4b54-85ef-1876fd51e106 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:34.212251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 661/1296] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:40:34.217534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc842c8-31fa-4cc9-83ef-5845f8807bd0 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:34.218127 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc842c8-31fa-4cc9-83ef-5845f8807bd0 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:34.218127 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc842c8-31fa-4cc9-83ef-5845f8807bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:34.237653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-ffc842c8-31fa-4cc9-83ef-5845f8807bd0 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:40:34.238855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ffc842c8-31fa-4cc9-83ef-5845f8807bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:40:34.310327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 636/1297] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:40:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:40:34.336651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62bcb0-2710-4b54-85ef-1876fd51e106 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:34.445648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62bcb0-2710-4b54-85ef-1876fd51e106 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:34.480536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5de4b2-15b2-4bfe-9ac9-50f5785301d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:34.481171 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5de4b2-15b2-4bfe-9ac9-50f5785301d0 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:34.481447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5de4b2-15b2-4bfe-9ac9-50f5785301d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:34.481714 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5de4b2-15b2-4bfe-9ac9-50f5785301d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:34.482337 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5de4b2-15b2-4bfe-9ac9-50f5785301d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:34.483736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5de4b2-15b2-4bfe-9ac9-50f5785301d0 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-ca5de4b2-15b2-4bfe-9ac9-50f5785301d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:34.492651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:34.493832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:34.495176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 662/1298] 10.222.0.86 () {56 vars in 1117 bytes} [Fri Aug 5 19:40:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:34.499800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc842c8-31fa-4cc9-83ef-5845f8807bd0 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:34.519161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:34.603300 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc842c8-31fa-4cc9-83ef-5845f8807bd0 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:34.637624 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92c26b-69f2-45a3-9923-2043ba1263f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:34.638229 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92c26b-69f2-45a3-9923-2043ba1263f8 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:34.638507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92c26b-69f2-45a3-9923-2043ba1263f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:34.638808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92c26b-69f2-45a3-9923-2043ba1263f8 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:34.639474 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92c26b-69f2-45a3-9923-2043ba1263f8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:34.639777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92c26b-69f2-45a3-9923-2043ba1263f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:34.641825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92c26b-69f2-45a3-9923-2043ba1263f8 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:34.647078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 637/1299] 10.222.0.86 () {56 vars in 1116 bytes} [Fri Aug 5 19:40:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:34.653073 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5de4b2-15b2-4bfe-9ac9-50f5785301d0 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:34.684132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:34.753956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5de4b2-15b2-4bfe-9ac9-50f5785301d0 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:34.790674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f58ac0-19c9-41e3-a1c5-f4df1aad284c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:34.791282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f58ac0-19c9-41e3-a1c5-f4df1aad284c None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:34.791558 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f58ac0-19c9-41e3-a1c5-f4df1aad284c None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:34.791827 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f58ac0-19c9-41e3-a1c5-f4df1aad284c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:34.792569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f58ac0-19c9-41e3-a1c5-f4df1aad284c None admin] RBAC: Adding query filter params (interface=public, service_id=654e545deb0541dcbf1567c0ad254495, region_id=RegionOne) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:34.792878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f58ac0-19c9-41e3-a1c5-f4df1aad284c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:34.794235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f58ac0-19c9-41e3-a1c5-f4df1aad284c None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:34.799913 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 663/1300] 10.222.0.86 () {56 vars in 1251 bytes} [Fri Aug 5 19:40:34 2022] GET /identity/v3/endpoints?service_id=654e545deb0541dcbf1567c0ad254495&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:34.805689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92c26b-69f2-45a3-9923-2043ba1263f8 None admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:34.888307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:34.912227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92c26b-69f2-45a3-9923-2043ba1263f8 None admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:34.951670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be1545-f3be-46bd-b71d-55d949b28052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:34.952338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be1545-f3be-46bd-b71d-55d949b28052 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:34.952761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be1545-f3be-46bd-b71d-55d949b28052 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:34.953060 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be1545-f3be-46bd-b71d-55d949b28052 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:34.955769 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be1545-f3be-46bd-b71d-55d949b28052 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:34.957224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be1545-f3be-46bd-b71d-55d949b28052 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:34.966512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 638/1301] 10.222.0.86 () {56 vars in 1087 bytes} [Fri Aug 5 19:40:34 2022] GET /identity/v3/services => generated 2191 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:40:36.972934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2808d-f8f8-4762-bd61-f82574c78d66 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:36.973520 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2808d-f8f8-4762-bd61-f82574c78d66 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:36.973520 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2808d-f8f8-4762-bd61-f82574c78d66 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:36.974887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 664/1302] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:40:36.980372 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03dfbc-1259-4ce4-bc80-8f8176bcc0cb None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:36.980829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03dfbc-1259-4ce4-bc80-8f8176bcc0cb None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:36.980919 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03dfbc-1259-4ce4-bc80-8f8176bcc0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:36.999627 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-ec03dfbc-1259-4ce4-bc80-8f8176bcc0cb None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:40:37.001559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ec03dfbc-1259-4ce4-bc80-8f8176bcc0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:40:37.077869 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 639/1303] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:40:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:40:37.101592 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2808d-f8f8-4762-bd61-f82574c78d66 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:37.211120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:37.218577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2808d-f8f8-4762-bd61-f82574c78d66 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:37.255095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c717cf2-0701-4e27-9725-f1c9211b43f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:37.255640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c717cf2-0701-4e27-9725-f1c9211b43f6 None admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:37.255913 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c717cf2-0701-4e27-9725-f1c9211b43f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:37.256179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c717cf2-0701-4e27-9725-f1c9211b43f6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:37.256796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c717cf2-0701-4e27-9725-f1c9211b43f6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:37.258139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c717cf2-0701-4e27-9725-f1c9211b43f6 None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application [None req-6c717cf2-0701-4e27-9725-f1c9211b43f6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:40:37.267319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:40:37.268770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application with Lock( Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: ERROR keystone.server.flask.application Aug 05 19:40:37.269828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 665/1304] 10.222.0.86 () {56 vars in 1117 bytes} [Fri Aug 5 19:40:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:40:37.275556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03dfbc-1259-4ce4-bc80-8f8176bcc0cb None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:37.355016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:37.380166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03dfbc-1259-4ce4-bc80-8f8176bcc0cb None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:37.420079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36823a21-accb-4d79-a2e4-136cfa6d27a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:37.420699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36823a21-accb-4d79-a2e4-136cfa6d27a7 None admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:37.420976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36823a21-accb-4d79-a2e4-136cfa6d27a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:37.421247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36823a21-accb-4d79-a2e4-136cfa6d27a7 None admin] PATH_INFO: `/v3/services` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:37.421947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36823a21-accb-4d79-a2e4-136cfa6d27a7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:40:37.422251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36823a21-accb-4d79-a2e4-136cfa6d27a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:37.423662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36823a21-accb-4d79-a2e4-136cfa6d27a7 None admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:37.429069 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 640/1305] 10.222.0.86 () {56 vars in 1116 bytes} [Fri Aug 5 19:40:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:40:37.434879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c717cf2-0701-4e27-9725-f1c9211b43f6 None admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:40:37.538749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c717cf2-0701-4e27-9725-f1c9211b43f6 None admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:40:37.572749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed7b69-c074-413d-91e1-705488e859ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:40:37.573352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed7b69-c074-413d-91e1-705488e859ab None admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:40:37.573625 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed7b69-c074-413d-91e1-705488e859ab None admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:40:37.573891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed7b69-c074-413d-91e1-705488e859ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:40:37.574600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed7b69-c074-413d-91e1-705488e859ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:40:37.575970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed7b69-c074-413d-91e1-705488e859ab None admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:40:37.598469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 666/1306] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:40:37 2022] POST /identity/v3/endpoints => generated 331 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:40:47.425798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=30 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:40:47.593040 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:41:42.251022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44611c1-c9df-4180-aff7-99471a776f7b None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:42.251796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44611c1-c9df-4180-aff7-99471a776f7b None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:42.251796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44611c1-c9df-4180-aff7-99471a776f7b None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:42.256768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 641/1307] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:41:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:41:42.258140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55aa22-9e13-47fb-9fd4-47b8cca3de35 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:42.258555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55aa22-9e13-47fb-9fd4-47b8cca3de35 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:42.258941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55aa22-9e13-47fb-9fd4-47b8cca3de35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:42.445997 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-fd55aa22-9e13-47fb-9fd4-47b8cca3de35 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:41:42.450160 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fd55aa22-9e13-47fb-9fd4-47b8cca3de35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:42.638739 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 667/1308] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:41:42.679549 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee69a6e-f77f-42df-b36d-82c4085cf2d5 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:42.680357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee69a6e-f77f-42df-b36d-82c4085cf2d5 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:42.680995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee69a6e-f77f-42df-b36d-82c4085cf2d5 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:42.683688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 642/1309] 10.222.0.86 () {54 vars in 965 bytes} [Fri Aug 5 19:41:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:41:42.690414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d991110-f738-428e-b833-aabeda100abb None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:42.690757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d991110-f738-428e-b833-aabeda100abb None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:42.691006 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d991110-f738-428e-b833-aabeda100abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:42.779421 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-8d991110-f738-428e-b833-aabeda100abb None None] MFA Rules not processed for user `80097afc3ce540e6a8060f696535e62c`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:41:42.780749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8d991110-f738-428e-b833-aabeda100abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:42.806114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:41:42.877488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 668/1310] 10.222.0.86 () {58 vars in 1062 bytes} [Fri Aug 5 19:41:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:41:42.888723 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee69a6e-f77f-42df-b36d-82c4085cf2d5 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:41:42.891115 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-aee69a6e-f77f-42df-b36d-82c4085cf2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:43.031671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee69a6e-f77f-42df-b36d-82c4085cf2d5 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:41:43.119319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c79bb-08b9-48a6-8c88-dbf1ce26b314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:41:43.119937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c79bb-08b9-48a6-8c88-dbf1ce26b314 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:43.120218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c79bb-08b9-48a6-8c88-dbf1ce26b314 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:43.120491 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c79bb-08b9-48a6-8c88-dbf1ce26b314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:43.122120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5d6c79bb-08b9-48a6-8c88-dbf1ce26b314 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:43.182834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:41:43.247302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c79bb-08b9-48a6-8c88-dbf1ce26b314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:41:43.249822 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c79bb-08b9-48a6-8c88-dbf1ce26b314 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:41:43.272294 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 643/1311] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:41:45.655027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b90f0-972b-4ea6-b9ff-bce0ccde88b0 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:45.655701 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b90f0-972b-4ea6-b9ff-bce0ccde88b0 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:45.655701 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b90f0-972b-4ea6-b9ff-bce0ccde88b0 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:45.657001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 669/1312] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:41:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:41:45.662847 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c2830-a700-4f69-bfc4-2dd79d4908a8 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:45.663343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c2830-a700-4f69-bfc4-2dd79d4908a8 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:45.663551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c2830-a700-4f69-bfc4-2dd79d4908a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:45.752647 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-6a1c2830-a700-4f69-bfc4-2dd79d4908a8 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:41:45.753943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6a1c2830-a700-4f69-bfc4-2dd79d4908a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:45.855004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 644/1313] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:41:45.893356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b90f0-972b-4ea6-b9ff-bce0ccde88b0 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:41:45.895004 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-cb0b90f0-972b-4ea6-b9ff-bce0ccde88b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:46.007312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b90f0-972b-4ea6-b9ff-bce0ccde88b0 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:41:46.078735 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:41:46.081510 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48004176-d8bc-4884-8ec0-797ab19a5bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:41:46.082370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48004176-d8bc-4884-8ec0-797ab19a5bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:46.082733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48004176-d8bc-4884-8ec0-797ab19a5bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:46.083209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48004176-d8bc-4884-8ec0-797ab19a5bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:46.085027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-48004176-d8bc-4884-8ec0-797ab19a5bb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:46.194556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48004176-d8bc-4884-8ec0-797ab19a5bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:41:46.196014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48004176-d8bc-4884-8ec0-797ab19a5bb0 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:41:46.214850 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 670/1314] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:41:48.575166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68969374-43ab-4bfb-b0ae-f4fc5085a703 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:48.576237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68969374-43ab-4bfb-b0ae-f4fc5085a703 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:48.576237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68969374-43ab-4bfb-b0ae-f4fc5085a703 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:48.577342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 645/1315] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:41:48.582763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5af88d-e9dc-45d0-ac45-457e6ed56914 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:48.583189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5af88d-e9dc-45d0-ac45-457e6ed56914 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:48.583311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5af88d-e9dc-45d0-ac45-457e6ed56914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:48.648392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-ba5af88d-e9dc-45d0-ac45-457e6ed56914 None None] MFA Rules not processed for user `4b27046155b1457883bf610e3be6b70e`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:41:48.649604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ba5af88d-e9dc-45d0-ac45-457e6ed56914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:48.702163 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:41:48.747361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 671/1316] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:41:48 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:41:48.800530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8c0b-a1ad-4c8c-a840-92ad36110eda None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:48.800809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8c0b-a1ad-4c8c-a840-92ad36110eda None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:48.801078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8c0b-a1ad-4c8c-a840-92ad36110eda None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:48.802418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 646/1317] 10.222.0.86 () {54 vars in 965 bytes} [Fri Aug 5 19:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:41:48.808066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c1882-931c-48c6-8e71-b79901a15827 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:48.808357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c1882-931c-48c6-8e71-b79901a15827 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:48.808631 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c1882-931c-48c6-8e71-b79901a15827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:48.886205 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-506c1882-931c-48c6-8e71-b79901a15827 None None] MFA Rules not processed for user `80097afc3ce540e6a8060f696535e62c`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:41:48.887722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-506c1882-931c-48c6-8e71-b79901a15827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:48.963924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 672/1318] 10.222.0.86 () {58 vars in 1062 bytes} [Fri Aug 5 19:41:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:41:48.973384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb8c0b-a1ad-4c8c-a840-92ad36110eda None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:41:48.974937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-0beb8c0b-a1ad-4c8c-a840-92ad36110eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:48.985408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:41:49.078725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb8c0b-a1ad-4c8c-a840-92ad36110eda None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:41:49.152580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e536ceb-b3cc-4b8a-b37b-96198f26ebac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:41:49.153208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e536ceb-b3cc-4b8a-b37b-96198f26ebac service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:49.153480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e536ceb-b3cc-4b8a-b37b-96198f26ebac service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:49.154791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e536ceb-b3cc-4b8a-b37b-96198f26ebac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:49.156415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-0e536ceb-b3cc-4b8a-b37b-96198f26ebac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:49.218381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:41:49.265678 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e536ceb-b3cc-4b8a-b37b-96198f26ebac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:41:49.267147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e536ceb-b3cc-4b8a-b37b-96198f26ebac service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:41:49.286122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 647/1319] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:41:48 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:41:52.325791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e12d1e-a81e-4884-a855-ed174c095e38 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:52.326418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e12d1e-a81e-4884-a855-ed174c095e38 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:52.326418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e12d1e-a81e-4884-a855-ed174c095e38 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:52.327720 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 673/1320] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:41:52.332895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191da0c-3474-4a98-992f-0be62c870741 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:52.333322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191da0c-3474-4a98-992f-0be62c870741 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:52.333489 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191da0c-3474-4a98-992f-0be62c870741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:52.402039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-4191da0c-3474-4a98-992f-0be62c870741 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:41:52.403273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-4191da0c-3474-4a98-992f-0be62c870741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:52.501453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 648/1321] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:41:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:41:52.541687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e12d1e-a81e-4884-a855-ed174c095e38 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:41:52.545126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-95e12d1e-a81e-4884-a855-ed174c095e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:52.599152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:41:52.651706 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e12d1e-a81e-4884-a855-ed174c095e38 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:41:52.725889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba545f-cb05-486c-aeaa-f729bf8f358f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:41:52.726493 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba545f-cb05-486c-aeaa-f729bf8f358f service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:52.726797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba545f-cb05-486c-aeaa-f729bf8f358f service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:52.727063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba545f-cb05-486c-aeaa-f729bf8f358f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:52.728675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c1ba545f-cb05-486c-aeaa-f729bf8f358f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:52.823857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:41:52.838119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba545f-cb05-486c-aeaa-f729bf8f358f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:41:52.839565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba545f-cb05-486c-aeaa-f729bf8f358f service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:41:52.859456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 674/1322] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:41:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:41:55.641405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7168a-898a-466d-9744-fa849cb0f9d3 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:55.641925 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7168a-898a-466d-9744-fa849cb0f9d3 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:55.641925 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7168a-898a-466d-9744-fa849cb0f9d3 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:55.643308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 649/1323] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:41:55.648657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e40ea1-5aec-4639-9c61-c1ff282e5090 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:55.649093 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e40ea1-5aec-4639-9c61-c1ff282e5090 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:55.649201 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e40ea1-5aec-4639-9c61-c1ff282e5090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:55.721143 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-89e40ea1-5aec-4639-9c61-c1ff282e5090 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:41:55.722328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-89e40ea1-5aec-4639-9c61-c1ff282e5090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:55.832105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 675/1324] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:41:55.870194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7168a-898a-466d-9744-fa849cb0f9d3 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:41:55.871769 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ddf7168a-898a-466d-9744-fa849cb0f9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:55.898043 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:41:55.994476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7168a-898a-466d-9744-fa849cb0f9d3 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:41:56.069835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6fcf0e-978c-45a6-928f-842013850cba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:41:56.070456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fcf0e-978c-45a6-928f-842013850cba service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:56.070768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fcf0e-978c-45a6-928f-842013850cba service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:56.071044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fcf0e-978c-45a6-928f-842013850cba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:56.072701 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-4d6fcf0e-978c-45a6-928f-842013850cba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:56.149500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:41:56.186503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fcf0e-978c-45a6-928f-842013850cba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:41:56.187895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fcf0e-978c-45a6-928f-842013850cba service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:41:56.206451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 650/1325] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:41:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:41:58.377581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cf4fb-1c06-4e93-902b-f4907dd7ca29 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:58.378121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cf4fb-1c06-4e93-902b-f4907dd7ca29 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:58.378121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cf4fb-1c06-4e93-902b-f4907dd7ca29 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:58.379432 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 676/1326] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:41:58.384663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca379e3-bc24-4647-83d0-655cb0ba85fb None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:58.385085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca379e3-bc24-4647-83d0-655cb0ba85fb None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:58.385204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca379e3-bc24-4647-83d0-655cb0ba85fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:58.457074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-cca379e3-bc24-4647-83d0-655cb0ba85fb None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:41:58.458292 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-cca379e3-bc24-4647-83d0-655cb0ba85fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:58.562432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 651/1327] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:41:58.598767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cf4fb-1c06-4e93-902b-f4907dd7ca29 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:41:58.600298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-106cf4fb-1c06-4e93-902b-f4907dd7ca29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:58.660986 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:41:58.706411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cf4fb-1c06-4e93-902b-f4907dd7ca29 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:41:58.786187 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa598f-d0ff-4d07-a874-bd16cd883dcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:41:58.786826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa598f-d0ff-4d07-a874-bd16cd883dcf service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:41:58.787106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa598f-d0ff-4d07-a874-bd16cd883dcf service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:41:58.787373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa598f-d0ff-4d07-a874-bd16cd883dcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:41:58.789002 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-beaa598f-d0ff-4d07-a874-bd16cd883dcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:41:58.898293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:41:58.900819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa598f-d0ff-4d07-a874-bd16cd883dcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:41:58.902270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa598f-d0ff-4d07-a874-bd16cd883dcf service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:41:58.927253 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 677/1328] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:41:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:01.036398 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56eac6-465b-4f61-acbb-a5110d5a7425 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:01.036974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56eac6-465b-4f61-acbb-a5110d5a7425 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:01.036974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf56eac6-465b-4f61-acbb-a5110d5a7425 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:01.038386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 652/1329] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:01.043588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b2b70-d351-473c-ac11-d707aec34969 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:01.044540 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b2b70-d351-473c-ac11-d707aec34969 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:01.044655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b2b70-d351-473c-ac11-d707aec34969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:01.116912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-237b2b70-d351-473c-ac11-d707aec34969 None None] MFA Rules not processed for user `4b27046155b1457883bf610e3be6b70e`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:01.118127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-237b2b70-d351-473c-ac11-d707aec34969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:01.230059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 678/1330] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:01 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:01.269736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56eac6-465b-4f61-acbb-a5110d5a7425 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:01.271398 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-bf56eac6-465b-4f61-acbb-a5110d5a7425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:01.302937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:01.381576 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf56eac6-465b-4f61-acbb-a5110d5a7425 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:01.459348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c8374-0815-4289-93d6-fe9a4041f1e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:01.459956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c8374-0815-4289-93d6-fe9a4041f1e5 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:01.460230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c8374-0815-4289-93d6-fe9a4041f1e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:01.460494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c8374-0815-4289-93d6-fe9a4041f1e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:01.462095 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-756c8374-0815-4289-93d6-fe9a4041f1e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:01.547850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:01.608577 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c8374-0815-4289-93d6-fe9a4041f1e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:01.610027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c8374-0815-4289-93d6-fe9a4041f1e5 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:01.630234 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 653/1331] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:01 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:03.917377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03790e8-959a-4ef5-ab95-8f573e18d446 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:03.917952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03790e8-959a-4ef5-ab95-8f573e18d446 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:03.917952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03790e8-959a-4ef5-ab95-8f573e18d446 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:03.919930 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 679/1332] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:03.925535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910539d-88ac-4402-97ca-bbb91e117dd7 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:03.925958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910539d-88ac-4402-97ca-bbb91e117dd7 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:03.926077 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910539d-88ac-4402-97ca-bbb91e117dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:03.994804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-a910539d-88ac-4402-97ca-bbb91e117dd7 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:03.996033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a910539d-88ac-4402-97ca-bbb91e117dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:04.096011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 654/1333] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:04.132671 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03790e8-959a-4ef5-ab95-8f573e18d446 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:04.134256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e03790e8-959a-4ef5-ab95-8f573e18d446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:04.205545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:04.240200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03790e8-959a-4ef5-ab95-8f573e18d446 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:04.312665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71b4af-5f53-4228-ad89-f5445f4294bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:04.313270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b4af-5f53-4228-ad89-f5445f4294bb service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:04.313539 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b4af-5f53-4228-ad89-f5445f4294bb service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:04.313811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b4af-5f53-4228-ad89-f5445f4294bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:04.315460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3a71b4af-5f53-4228-ad89-f5445f4294bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:04.424644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b4af-5f53-4228-ad89-f5445f4294bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:04.426071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b4af-5f53-4228-ad89-f5445f4294bb service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:04.436849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:04.446899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 680/1334] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:06.507245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8d6fe-2a4a-4b88-b7eb-e423a8f83780 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:06.507770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8d6fe-2a4a-4b88-b7eb-e423a8f83780 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:06.507770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8d6fe-2a4a-4b88-b7eb-e423a8f83780 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:06.509136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 655/1335] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:06.515565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15151116-5079-4d77-83e4-9e629b3b1be3 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:06.516050 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15151116-5079-4d77-83e4-9e629b3b1be3 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:06.516148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15151116-5079-4d77-83e4-9e629b3b1be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:06.581834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-15151116-5079-4d77-83e4-9e629b3b1be3 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:06.583081 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-15151116-5079-4d77-83e4-9e629b3b1be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:06.678549 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 681/1336] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:06.720348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8d6fe-2a4a-4b88-b7eb-e423a8f83780 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:06.721885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3ad8d6fe-2a4a-4b88-b7eb-e423a8f83780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:06.762799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:06.828807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8d6fe-2a4a-4b88-b7eb-e423a8f83780 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:06.900906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69616a4a-4043-46b3-bbe9-10be73263e46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:06.901513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69616a4a-4043-46b3-bbe9-10be73263e46 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:06.901785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69616a4a-4043-46b3-bbe9-10be73263e46 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:06.902053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69616a4a-4043-46b3-bbe9-10be73263e46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:06.903987 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-69616a4a-4043-46b3-bbe9-10be73263e46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:06.988434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:07.015663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69616a4a-4043-46b3-bbe9-10be73263e46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:07.017025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69616a4a-4043-46b3-bbe9-10be73263e46 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:07.035210 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 656/1337] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:09.565855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b3c8b-d9f1-4887-91ac-b416a487d28c None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:09.566391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b3c8b-d9f1-4887-91ac-b416a487d28c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:09.566391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b3c8b-d9f1-4887-91ac-b416a487d28c None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:09.567813 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 682/1338] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:09.573023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935653de-1b1c-4027-ba8e-2e3e0e31c92f None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:09.573417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935653de-1b1c-4027-ba8e-2e3e0e31c92f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:09.573580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935653de-1b1c-4027-ba8e-2e3e0e31c92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:09.642115 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-935653de-1b1c-4027-ba8e-2e3e0e31c92f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:09.643380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-935653de-1b1c-4027-ba8e-2e3e0e31c92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:09.738263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 657/1339] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:09.774503 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b3c8b-d9f1-4887-91ac-b416a487d28c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:09.776045 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-512b3c8b-d9f1-4887-91ac-b416a487d28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:09.852223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:09.881828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b3c8b-d9f1-4887-91ac-b416a487d28c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:09.961001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c89535-494a-4b32-ae30-b20dc8f5cf49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:09.961615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c89535-494a-4b32-ae30-b20dc8f5cf49 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:09.961892 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c89535-494a-4b32-ae30-b20dc8f5cf49 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:09.962158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c89535-494a-4b32-ae30-b20dc8f5cf49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:09.963816 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-22c89535-494a-4b32-ae30-b20dc8f5cf49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:10.072425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c89535-494a-4b32-ae30-b20dc8f5cf49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:10.073853 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c89535-494a-4b32-ae30-b20dc8f5cf49 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:10.092569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 683/1340] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:14.310239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee662db0-2a3d-4402-ba10-61e151367265 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:14.311424 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee662db0-2a3d-4402-ba10-61e151367265 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:14.311424 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee662db0-2a3d-4402-ba10-61e151367265 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:14.312594 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 658/1341] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:14.317973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5f60f-5587-49b5-b921-c78d293fb773 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:14.318488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5f60f-5587-49b5-b921-c78d293fb773 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:14.318837 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5f60f-5587-49b5-b921-c78d293fb773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:14.325872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:14.390375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e1e5f60f-5587-49b5-b921-c78d293fb773 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:14.391545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e1e5f60f-5587-49b5-b921-c78d293fb773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:14.500614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 684/1342] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:14.539718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee662db0-2a3d-4402-ba10-61e151367265 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:14.541229 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ee662db0-2a3d-4402-ba10-61e151367265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:14.591672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:14.649133 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee662db0-2a3d-4402-ba10-61e151367265 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:14.723396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdad727-d15f-4829-bd9c-e08b8943eb34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:14.724006 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdad727-d15f-4829-bd9c-e08b8943eb34 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:14.724285 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdad727-d15f-4829-bd9c-e08b8943eb34 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:14.724553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdad727-d15f-4829-bd9c-e08b8943eb34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:14.726172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-4fdad727-d15f-4829-bd9c-e08b8943eb34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:14.821903 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:14.838529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdad727-d15f-4829-bd9c-e08b8943eb34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:14.840588 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdad727-d15f-4829-bd9c-e08b8943eb34 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:14.859690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 659/1343] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:42:17.558340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ed687-0a16-47f0-a24f-8ae4e849d01c None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:17.559463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ed687-0a16-47f0-a24f-8ae4e849d01c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:17.559463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ed687-0a16-47f0-a24f-8ae4e849d01c None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:17.561270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 685/1344] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:17.567083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f607053-8d77-4832-8964-1011a4943131 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:17.567629 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f607053-8d77-4832-8964-1011a4943131 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:17.567629 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f607053-8d77-4832-8964-1011a4943131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:17.640176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-7f607053-8d77-4832-8964-1011a4943131 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:17.641395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7f607053-8d77-4832-8964-1011a4943131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:17.744073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 660/1345] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:17.782315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ed687-0a16-47f0-a24f-8ae4e849d01c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:17.784053 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6c6ed687-0a16-47f0-a24f-8ae4e849d01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:17.872242 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:17.890198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ed687-0a16-47f0-a24f-8ae4e849d01c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:17.967129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045aa9e-9ea2-44dc-ade7-4eb98dc506bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:17.967748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045aa9e-9ea2-44dc-ade7-4eb98dc506bd service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:17.968024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045aa9e-9ea2-44dc-ade7-4eb98dc506bd service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:17.968292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045aa9e-9ea2-44dc-ade7-4eb98dc506bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:17.969916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4045aa9e-9ea2-44dc-ade7-4eb98dc506bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:18.084595 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045aa9e-9ea2-44dc-ade7-4eb98dc506bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:18.086023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045aa9e-9ea2-44dc-ade7-4eb98dc506bd service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:18.107872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 686/1346] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:22.204905 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaca5f-17e1-49d3-a671-d6456d7478ac None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:22.205468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaca5f-17e1-49d3-a671-d6456d7478ac None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:22.205468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaca5f-17e1-49d3-a671-d6456d7478ac None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:22.206968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 661/1347] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:22.212316 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b665231-6179-4749-8316-5188bd8d4fd7 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:22.212767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b665231-6179-4749-8316-5188bd8d4fd7 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:22.212858 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b665231-6179-4749-8316-5188bd8d4fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:22.229542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:22.284604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3b665231-6179-4749-8316-5188bd8d4fd7 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:22.285729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3b665231-6179-4749-8316-5188bd8d4fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:22.412330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 687/1348] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:42:22.452421 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eaca5f-17e1-49d3-a671-d6456d7478ac None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:22.454034 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-18eaca5f-17e1-49d3-a671-d6456d7478ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:22.521666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:22.571681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eaca5f-17e1-49d3-a671-d6456d7478ac None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:22.654498 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a45999-4062-45b8-8cb5-c83ecb1b518d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:22.655154 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a45999-4062-45b8-8cb5-c83ecb1b518d service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:22.655434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a45999-4062-45b8-8cb5-c83ecb1b518d service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:22.655709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a45999-4062-45b8-8cb5-c83ecb1b518d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:22.657362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-77a45999-4062-45b8-8cb5-c83ecb1b518d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:22.772371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:22.776641 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a45999-4062-45b8-8cb5-c83ecb1b518d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:22.779107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a45999-4062-45b8-8cb5-c83ecb1b518d service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:22.799246 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 662/1349] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:27.918450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7b585-b796-470b-b247-270566a5d477 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:27.919080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7b585-b796-470b-b247-270566a5d477 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:27.919080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7b585-b796-470b-b247-270566a5d477 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:27.920624 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 688/1350] 10.222.0.86 () {54 vars in 921 bytes} [Fri Aug 5 19:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:27.927055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c00de-72ca-4136-954e-9ee47ec81c73 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:27.927928 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c00de-72ca-4136-954e-9ee47ec81c73 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:27.928412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c00de-72ca-4136-954e-9ee47ec81c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:28.076245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-a43c00de-72ca-4136-954e-9ee47ec81c73 None None] MFA Rules not processed for user `cce71f93c80d433b830c448d0a7a14e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:28.077516 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a43c00de-72ca-4136-954e-9ee47ec81c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:28.215346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 663/1351] 10.222.0.86 () {58 vars in 1018 bytes} [Fri Aug 5 19:42:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:32.376488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2656e2-2ff9-4ebc-b277-4e82b5fba0e9 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:32.377015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2656e2-2ff9-4ebc-b277-4e82b5fba0e9 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:32.377015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2656e2-2ff9-4ebc-b277-4e82b5fba0e9 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:32.378649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 689/1352] 10.222.0.86 () {54 vars in 921 bytes} [Fri Aug 5 19:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:32.384188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5e557-354f-4c1b-8eac-165b9ba3e46d None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:32.384661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5e557-354f-4c1b-8eac-165b9ba3e46d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:32.384944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5e557-354f-4c1b-8eac-165b9ba3e46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:32.410195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=59 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:32.448438 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:32.474981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-acc5e557-354f-4c1b-8eac-165b9ba3e46d None None] MFA Rules not processed for user `cce71f93c80d433b830c448d0a7a14e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:32.476208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-acc5e557-354f-4c1b-8eac-165b9ba3e46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:32.599170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 664/1353] 10.222.0.86 () {58 vars in 1018 bytes} [Fri Aug 5 19:42:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:33.977279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ffcb5-709c-4bce-8441-51d7e2f75639 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:33.977792 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ffcb5-709c-4bce-8441-51d7e2f75639 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:33.977893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ffcb5-709c-4bce-8441-51d7e2f75639 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:33.979282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 690/1354] 10.222.0.86 () {54 vars in 921 bytes} [Fri Aug 5 19:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:33.985520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad181442-d861-4e0f-8937-a2572ac00c41 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:33.986019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad181442-d861-4e0f-8937-a2572ac00c41 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:33.986145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad181442-d861-4e0f-8937-a2572ac00c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:34.075182 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-ad181442-d861-4e0f-8937-a2572ac00c41 None None] MFA Rules not processed for user `cce71f93c80d433b830c448d0a7a14e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:34.076395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ad181442-d861-4e0f-8937-a2572ac00c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:34.121541 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:34.189463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 665/1355] 10.222.0.86 () {58 vars in 1018 bytes} [Fri Aug 5 19:42:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:42:37.822860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301c0e-87c3-4b46-b4ca-f3cc43d206a5 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:37.824531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301c0e-87c3-4b46-b4ca-f3cc43d206a5 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:37.825232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301c0e-87c3-4b46-b4ca-f3cc43d206a5 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:37.829913 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 691/1356] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:37.832554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3aff2c-05ea-4bd0-9361-61b3d14e91dc None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:37.833021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3aff2c-05ea-4bd0-9361-61b3d14e91dc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:37.833132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3aff2c-05ea-4bd0-9361-61b3d14e91dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:37.910583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-7d3aff2c-05ea-4bd0-9361-61b3d14e91dc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:37.913698 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7d3aff2c-05ea-4bd0-9361-61b3d14e91dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:38.020991 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 666/1357] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:38.078672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f814f71-a901-40aa-a28b-9efb5fe11659 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:38.079392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f814f71-a901-40aa-a28b-9efb5fe11659 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:38.080139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f814f71-a901-40aa-a28b-9efb5fe11659 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:38.081917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 692/1358] 10.222.0.86 () {54 vars in 962 bytes} [Fri Aug 5 19:42:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:38.091562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090f18a-f70d-40a6-babb-5ded6e3a084d None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:38.092225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090f18a-f70d-40a6-babb-5ded6e3a084d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:38.092717 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090f18a-f70d-40a6-babb-5ded6e3a084d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:38.115691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:38.204998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c090f18a-f70d-40a6-babb-5ded6e3a084d None None] MFA Rules not processed for user `ab8f9d9cf4ca4e38ae6eb67e949b40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:38.206610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c090f18a-f70d-40a6-babb-5ded6e3a084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:38.346530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 667/1359] 10.222.0.86 () {58 vars in 1059 bytes} [Fri Aug 5 19:42:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:42:38.354589 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f814f71-a901-40aa-a28b-9efb5fe11659 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:38.356745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4f814f71-a901-40aa-a28b-9efb5fe11659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:38.525420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f814f71-a901-40aa-a28b-9efb5fe11659 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:38.607183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:38.652651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699b192-5541-4c0f-8a5f-8028b3980aaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:38.653269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699b192-5541-4c0f-8a5f-8028b3980aaa service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:38.653541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699b192-5541-4c0f-8a5f-8028b3980aaa service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:38.653805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699b192-5541-4c0f-8a5f-8028b3980aaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:38.655710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d699b192-5541-4c0f-8a5f-8028b3980aaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:38.751138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699b192-5541-4c0f-8a5f-8028b3980aaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:38.753009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699b192-5541-4c0f-8a5f-8028b3980aaa service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:38.772932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 693/1360] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:41.033312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3692f-1d5b-428a-a437-3da4cc7a22ac None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:41.033899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3692f-1d5b-428a-a437-3da4cc7a22ac None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:41.033899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3692f-1d5b-428a-a437-3da4cc7a22ac None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:41.035396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 668/1361] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:41.040489 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690898a-e919-4907-9990-6d31311503ac None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:41.040915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690898a-e919-4907-9990-6d31311503ac None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:41.041020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690898a-e919-4907-9990-6d31311503ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:41.115697 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-5690898a-e919-4907-9990-6d31311503ac None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:41.116893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5690898a-e919-4907-9990-6d31311503ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:41.159467 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:41.230763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 694/1362] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:41.278751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3692f-1d5b-428a-a437-3da4cc7a22ac None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:41.280259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f1e3692f-1d5b-428a-a437-3da4cc7a22ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:41.369080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:41.489579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3692f-1d5b-428a-a437-3da4cc7a22ac None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:41.643902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdd9c3-924e-444d-8c8b-da972bee0e7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:41.644555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdd9c3-924e-444d-8c8b-da972bee0e7d service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:41.644870 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdd9c3-924e-444d-8c8b-da972bee0e7d service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:41.645233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdd9c3-924e-444d-8c8b-da972bee0e7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:41.646920 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-52cdd9c3-924e-444d-8c8b-da972bee0e7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:41.673862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:41.758381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdd9c3-924e-444d-8c8b-da972bee0e7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:41.760096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdd9c3-924e-444d-8c8b-da972bee0e7d service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:41.780028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 669/1363] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:44.232301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02cb85-8d05-4862-b504-ed61a104486d None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:44.233907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02cb85-8d05-4862-b504-ed61a104486d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:44.234356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02cb85-8d05-4862-b504-ed61a104486d None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:44.239467 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 695/1364] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:44.246221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e3eea-b326-4a61-bc94-bebf9a19cc48 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:44.246221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e3eea-b326-4a61-bc94-bebf9a19cc48 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:44.246990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e3eea-b326-4a61-bc94-bebf9a19cc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:44.329490 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-2c0e3eea-b326-4a61-bc94-bebf9a19cc48 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:44.331270 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2c0e3eea-b326-4a61-bc94-bebf9a19cc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:44.453663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:44.462936 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 670/1365] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:44.512439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac14ed-1f1d-4369-94ca-c8fe943ed534 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:44.515113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac14ed-1f1d-4369-94ca-c8fe943ed534 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:44.515422 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac14ed-1f1d-4369-94ca-c8fe943ed534 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:44.516860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 696/1366] 10.222.0.86 () {54 vars in 962 bytes} [Fri Aug 5 19:42:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:44.522713 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58828c-83da-442a-ad17-543095618316 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:44.522998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58828c-83da-442a-ad17-543095618316 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:44.523271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58828c-83da-442a-ad17-543095618316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:44.619380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-fc58828c-83da-442a-ad17-543095618316 None None] MFA Rules not processed for user `ab8f9d9cf4ca4e38ae6eb67e949b40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:44.621481 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-fc58828c-83da-442a-ad17-543095618316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:44.758474 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 671/1367] 10.222.0.86 () {58 vars in 1059 bytes} [Fri Aug 5 19:42:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:44.767501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac14ed-1f1d-4369-94ca-c8fe943ed534 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:44.769143 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b7ac14ed-1f1d-4369-94ca-c8fe943ed534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:44.939999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac14ed-1f1d-4369-94ca-c8fe943ed534 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:44.962258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:45.067053 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81288037-5ccb-4567-94c2-a78b70027939 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:45.067673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288037-5ccb-4567-94c2-a78b70027939 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:45.067947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288037-5ccb-4567-94c2-a78b70027939 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:45.068213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81288037-5ccb-4567-94c2-a78b70027939 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:45.069853 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-81288037-5ccb-4567-94c2-a78b70027939 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:45.170161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81288037-5ccb-4567-94c2-a78b70027939 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:45.171798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81288037-5ccb-4567-94c2-a78b70027939 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:45.193279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 697/1368] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:46.182089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb7137-2743-43db-9dbf-b00f6d76b3d8 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:46.182681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb7137-2743-43db-9dbf-b00f6d76b3d8 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:46.182681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb7137-2743-43db-9dbf-b00f6d76b3d8 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:46.186674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 672/1369] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:46.190445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadfe21-ea98-4b3f-8a43-829570046b7c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:46.190925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadfe21-ea98-4b3f-8a43-829570046b7c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:46.191016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadfe21-ea98-4b3f-8a43-829570046b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:46.227915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:46.268148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-5cadfe21-ea98-4b3f-8a43-829570046b7c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:46.269946 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5cadfe21-ea98-4b3f-8a43-829570046b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:46.384182 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 698/1370] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:46.740075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fa836-04b7-4afd-bb73-073ff503ee10 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:46.740075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fa836-04b7-4afd-bb73-073ff503ee10 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:46.740075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fa836-04b7-4afd-bb73-073ff503ee10 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:46.741970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 673/1371] 10.222.0.86 () {54 vars in 964 bytes} [Fri Aug 5 19:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:46.746201 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c22a-ca38-4ad2-abfa-f52126f88dc6 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:46.746485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c22a-ca38-4ad2-abfa-f52126f88dc6 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:46.746789 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6c22a-ca38-4ad2-abfa-f52126f88dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:46.829560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-18e6c22a-ca38-4ad2-abfa-f52126f88dc6 None None] MFA Rules not processed for user `afea7d279292424490beff0481ff1512`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:46.830825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-18e6c22a-ca38-4ad2-abfa-f52126f88dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:46.880633 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:46.914387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 699/1372] 10.222.0.86 () {58 vars in 1061 bytes} [Fri Aug 5 19:42:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:46.924021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fa836-04b7-4afd-bb73-073ff503ee10 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:46.936362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-269fa836-04b7-4afd-bb73-073ff503ee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:46.998887 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:47.059560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269fa836-04b7-4afd-bb73-073ff503ee10 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:47.144853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcb008-ebb7-4431-bca4-58c8474cda8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:47.145475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcb008-ebb7-4431-bca4-58c8474cda8d service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:47.145753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcb008-ebb7-4431-bca4-58c8474cda8d service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:47.146022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcb008-ebb7-4431-bca4-58c8474cda8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:47.147792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f4fcb008-ebb7-4431-bca4-58c8474cda8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:47.254024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:47.272340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcb008-ebb7-4431-bca4-58c8474cda8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:47.273914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcb008-ebb7-4431-bca4-58c8474cda8d service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:47.297936 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 674/1373] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:47.311473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9fd1c-2992-4a3a-b432-78144a9c7ea8 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:47.311957 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9fd1c-2992-4a3a-b432-78144a9c7ea8 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:47.312059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9fd1c-2992-4a3a-b432-78144a9c7ea8 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:47.313549 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 700/1374] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:47.318996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1658-a244-4b77-bdde-825dc42c7965 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:47.319280 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1658-a244-4b77-bdde-825dc42c7965 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:47.319549 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1658-a244-4b77-bdde-825dc42c7965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:47.397469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-767d1658-a244-4b77-bdde-825dc42c7965 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:47.398711 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-767d1658-a244-4b77-bdde-825dc42c7965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:47.522396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 675/1375] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:47.572293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9fd1c-2992-4a3a-b432-78144a9c7ea8 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:47.574012 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-41b9fd1c-2992-4a3a-b432-78144a9c7ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:47.743665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9fd1c-2992-4a3a-b432-78144a9c7ea8 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:47.796490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:47.860801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ba81b-1714-4714-b163-a0816d9c8743 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:47.861414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ba81b-1714-4714-b163-a0816d9c8743 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:47.861698 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ba81b-1714-4714-b163-a0816d9c8743 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:47.861969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ba81b-1714-4714-b163-a0816d9c8743 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:47.863635 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-515ba81b-1714-4714-b163-a0816d9c8743 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:47.960734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ba81b-1714-4714-b163-a0816d9c8743 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:47.962352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ba81b-1714-4714-b163-a0816d9c8743 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:47.982258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 701/1376] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:50.095903 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a1caf-cfb6-41e7-81c6-5045d93d96e5 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:50.096450 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a1caf-cfb6-41e7-81c6-5045d93d96e5 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:50.096450 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a1caf-cfb6-41e7-81c6-5045d93d96e5 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:50.097790 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 676/1377] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:50.103004 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c521fc-bc5e-4116-ab6c-49d0e8438935 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:50.103438 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c521fc-bc5e-4116-ab6c-49d0e8438935 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:50.103760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c521fc-bc5e-4116-ab6c-49d0e8438935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:50.172048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-b9c521fc-bc5e-4116-ab6c-49d0e8438935 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:50.173261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b9c521fc-bc5e-4116-ab6c-49d0e8438935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:50.179841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:50.281013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 702/1378] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:50.326936 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a1caf-cfb6-41e7-81c6-5045d93d96e5 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:50.328448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f61a1caf-cfb6-41e7-81c6-5045d93d96e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:50.399771 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:50.505307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a1caf-cfb6-41e7-81c6-5045d93d96e5 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:50.623276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d2fef-d4e4-4018-8121-297afeaf9cc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:50.623916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d2fef-d4e4-4018-8121-297afeaf9cc7 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:50.624263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d2fef-d4e4-4018-8121-297afeaf9cc7 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:50.624603 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d2fef-d4e4-4018-8121-297afeaf9cc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:50.626335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-710d2fef-d4e4-4018-8121-297afeaf9cc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:50.645122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:50.724608 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d2fef-d4e4-4018-8121-297afeaf9cc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:50.726238 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d2fef-d4e4-4018-8121-297afeaf9cc7 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:50.746317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 677/1379] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:52.869005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5940d9-31fb-4171-9c97-22beff851781 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:52.869576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5940d9-31fb-4171-9c97-22beff851781 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:52.869576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5940d9-31fb-4171-9c97-22beff851781 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:52.872074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 703/1380] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:52.877362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb5b86-be74-43fc-b638-5c02637b1c07 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:52.877794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb5b86-be74-43fc-b638-5c02637b1c07 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:52.877913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb5b86-be74-43fc-b638-5c02637b1c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:52.954378 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-58cb5b86-be74-43fc-b638-5c02637b1c07 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:52.956665 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-58cb5b86-be74-43fc-b638-5c02637b1c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:53.067538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:53.074965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 678/1381] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:53.120643 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5940d9-31fb-4171-9c97-22beff851781 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:53.122778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2f5940d9-31fb-4171-9c97-22beff851781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:53.271544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:53.284912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5940d9-31fb-4171-9c97-22beff851781 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:53.413552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7250b20-7ece-463c-a140-ae1e8fae94a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:53.414083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7250b20-7ece-463c-a140-ae1e8fae94a0 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:53.414356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7250b20-7ece-463c-a140-ae1e8fae94a0 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:53.414644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7250b20-7ece-463c-a140-ae1e8fae94a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:53.416256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f7250b20-7ece-463c-a140-ae1e8fae94a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:53.522954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7250b20-7ece-463c-a140-ae1e8fae94a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:53.524673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7250b20-7ece-463c-a140-ae1e8fae94a0 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:53.536818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:53.547954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 704/1382] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:42:55.718306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d36c4-2140-44c0-9234-4ceccfafa9f1 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:55.718867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d36c4-2140-44c0-9234-4ceccfafa9f1 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:55.718950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d36c4-2140-44c0-9234-4ceccfafa9f1 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:55.720267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 679/1383] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:55.727128 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d112f57-0b96-4a20-9c9d-751bd9d761f7 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:55.727653 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d112f57-0b96-4a20-9c9d-751bd9d761f7 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:55.728021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d112f57-0b96-4a20-9c9d-751bd9d761f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:55.964464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-6d112f57-0b96-4a20-9c9d-751bd9d761f7 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:55.966116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6d112f57-0b96-4a20-9c9d-751bd9d761f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:56.082660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 705/1384] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:56.135560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d36c4-2140-44c0-9234-4ceccfafa9f1 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:56.137483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-741d36c4-2140-44c0-9234-4ceccfafa9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:56.315889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d36c4-2140-44c0-9234-4ceccfafa9f1 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:56.394574 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:56.446268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff0a14-2a3b-4a9a-aaa2-62791c1a4fdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:56.447023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff0a14-2a3b-4a9a-aaa2-62791c1a4fdd service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:56.447331 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff0a14-2a3b-4a9a-aaa2-62791c1a4fdd service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:56.447655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff0a14-2a3b-4a9a-aaa2-62791c1a4fdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:56.450297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d3ff0a14-2a3b-4a9a-aaa2-62791c1a4fdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:56.562952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff0a14-2a3b-4a9a-aaa2-62791c1a4fdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:56.564552 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff0a14-2a3b-4a9a-aaa2-62791c1a4fdd service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:56.583160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 680/1385] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:42:58.633911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327fc4b-8478-4136-86fa-11646c7db4ad None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:58.634647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327fc4b-8478-4136-86fa-11646c7db4ad None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:58.634647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327fc4b-8478-4136-86fa-11646c7db4ad None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:58.636559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 706/1386] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:42:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:42:58.642413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddbd95-b797-4f52-8b29-0cc3a3c2cc1e None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:58.643060 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddbd95-b797-4f52-8b29-0cc3a3c2cc1e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:58.643195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddbd95-b797-4f52-8b29-0cc3a3c2cc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:58.721126 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-4dddbd95-b797-4f52-8b29-0cc3a3c2cc1e None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:42:58.722349 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-4dddbd95-b797-4f52-8b29-0cc3a3c2cc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:58.756819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:58.833186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 681/1387] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:42:58.879983 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327fc4b-8478-4136-86fa-11646c7db4ad None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:42:58.881990 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4327fc4b-8478-4136-86fa-11646c7db4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:58.981456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:59.052028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327fc4b-8478-4136-86fa-11646c7db4ad None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:42:59.170840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc2b43-dca6-4649-92b1-03ec31f4df6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:42:59.171458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc2b43-dca6-4649-92b1-03ec31f4df6c service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:42:59.171855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc2b43-dca6-4649-92b1-03ec31f4df6c service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:42:59.172005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc2b43-dca6-4649-92b1-03ec31f4df6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:42:59.173621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c7bc2b43-dca6-4649-92b1-03ec31f4df6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:42:59.214841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:42:59.269536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc2b43-dca6-4649-92b1-03ec31f4df6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:42:59.271141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc2b43-dca6-4649-92b1-03ec31f4df6c service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:42:59.292249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 707/1388] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:01.525116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e2b39-8cc0-4c48-8908-6375f178af93 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:01.525715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e2b39-8cc0-4c48-8908-6375f178af93 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:01.525715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e2b39-8cc0-4c48-8908-6375f178af93 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:01.527131 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 682/1389] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:01.532731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615d068-70db-4f2b-8e2c-cd9edaebe729 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:01.533212 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615d068-70db-4f2b-8e2c-cd9edaebe729 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:01.533329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615d068-70db-4f2b-8e2c-cd9edaebe729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:01.606099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e615d068-70db-4f2b-8e2c-cd9edaebe729 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:01.607348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e615d068-70db-4f2b-8e2c-cd9edaebe729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:01.713811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 708/1390] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:01.767257 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e2b39-8cc0-4c48-8908-6375f178af93 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:01.769351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b58e2b39-8cc0-4c48-8908-6375f178af93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:01.939542 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e2b39-8cc0-4c48-8908-6375f178af93 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:01.957176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:02.072061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4b18f-b4a0-4d44-a9ef-2bd9fcd3f3a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:02.072722 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4b18f-b4a0-4d44-a9ef-2bd9fcd3f3a5 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:02.073307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4b18f-b4a0-4d44-a9ef-2bd9fcd3f3a5 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:02.073440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4b18f-b4a0-4d44-a9ef-2bd9fcd3f3a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:02.075039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3fd4b18f-b4a0-4d44-a9ef-2bd9fcd3f3a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:02.165406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4b18f-b4a0-4d44-a9ef-2bd9fcd3f3a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:02.167561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4b18f-b4a0-4d44-a9ef-2bd9fcd3f3a5 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:02.186567 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 683/1391] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:43:02.935648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cfc92-e4ea-4303-b0fe-8471a80f7737 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:02.936721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cfc92-e4ea-4303-b0fe-8471a80f7737 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:02.936882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cfc92-e4ea-4303-b0fe-8471a80f7737 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:02.938306 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 709/1392] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:02.945385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b9753-a49b-44dd-8be0-429100dec0fc None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:02.945913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b9753-a49b-44dd-8be0-429100dec0fc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:02.946287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b9753-a49b-44dd-8be0-429100dec0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:02.976326 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:03.020216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-730b9753-a49b-44dd-8be0-429100dec0fc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:03.021414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-730b9753-a49b-44dd-8be0-429100dec0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:03.117807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 684/1393] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:03.173533 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a08521-49c5-46db-9214-4c9e9ce08519 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:03.173812 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a08521-49c5-46db-9214-4c9e9ce08519 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:03.174081 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a08521-49c5-46db-9214-4c9e9ce08519 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:03.175545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 710/1394] 10.222.0.86 () {54 vars in 966 bytes} [Fri Aug 5 19:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:03.180775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f6dbf-b56f-4cf4-bc98-e51b776b18e0 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:03.181063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f6dbf-b56f-4cf4-bc98-e51b776b18e0 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:03.181333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f6dbf-b56f-4cf4-bc98-e51b776b18e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:03.260893 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-de0f6dbf-b56f-4cf4-bc98-e51b776b18e0 None None] MFA Rules not processed for user `72d4459164844e718544d7fde333db76`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:03.262068 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-de0f6dbf-b56f-4cf4-bc98-e51b776b18e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:03.302977 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:03.341993 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 685/1395] 10.222.0.86 () {58 vars in 1063 bytes} [Fri Aug 5 19:43:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:03.350198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a08521-49c5-46db-9214-4c9e9ce08519 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:03.351749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-57a08521-49c5-46db-9214-4c9e9ce08519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:03.366769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:03.462406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a08521-49c5-46db-9214-4c9e9ce08519 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:03.540771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24485f-3389-4d75-aca6-acd2bdd69e30 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72d4459164844e718544d7fde333db76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:03.541867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24485f-3389-4d75-aca6-acd2bdd69e30 service glance] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:03.542591 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24485f-3389-4d75-aca6-acd2bdd69e30 service glance] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:03.542827 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24485f-3389-4d75-aca6-acd2bdd69e30 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:03.544426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2b24485f-3389-4d75-aca6-acd2bdd69e30 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:03.616077 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:03.659762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24485f-3389-4d75-aca6-acd2bdd69e30 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:03.661197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24485f-3389-4d75-aca6-acd2bdd69e30 service glance] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:03.680488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 711/1396] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:04.329474 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4b575-6a3e-4faa-9341-1a48e567757a None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:04.330093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4b575-6a3e-4faa-9341-1a48e567757a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:04.330093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4b575-6a3e-4faa-9341-1a48e567757a None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:04.331685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 686/1397] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:04.337269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b058e9d-6d97-40fc-b82e-b3bef47ed4a2 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:04.337839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b058e9d-6d97-40fc-b82e-b3bef47ed4a2 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:04.337839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b058e9d-6d97-40fc-b82e-b3bef47ed4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:04.411771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-2b058e9d-6d97-40fc-b82e-b3bef47ed4a2 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:04.413108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2b058e9d-6d97-40fc-b82e-b3bef47ed4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:04.440207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0281e-2078-48b8-8724-76c3f98a713a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:04.440496 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0281e-2078-48b8-8724-76c3f98a713a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:04.440768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a0281e-2078-48b8-8724-76c3f98a713a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:04.513625 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-26a0281e-2078-48b8-8724-76c3f98a713a None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:04.515036 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-26a0281e-2078-48b8-8724-76c3f98a713a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:04.519819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 712/1398] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:04.565712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b058e9d-6d97-40fc-b82e-b3bef47ed4a2 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:04.567024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2b058e9d-6d97-40fc-b82e-b3bef47ed4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:04.595664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:04.609677 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 687/1399] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:43:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:43:04.658499 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a28367-98da-4768-bb36-658754ac9d45 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:04.658821 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a28367-98da-4768-bb36-658754ac9d45 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:04.659096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a28367-98da-4768-bb36-658754ac9d45 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:04.660448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 688/1400] 10.222.0.86 () {54 vars in 943 bytes} [Fri Aug 5 19:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:43:04.666884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b31c86-544b-4a99-baba-2ecf874cd26a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:04.667235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b31c86-544b-4a99-baba-2ecf874cd26a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:04.667517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b31c86-544b-4a99-baba-2ecf874cd26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:04.688767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b058e9d-6d97-40fc-b82e-b3bef47ed4a2 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:04.739168 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:04.748043 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-08b31c86-544b-4a99-baba-2ecf874cd26a None None] MFA Rules not processed for user `e8adb0fd1fbc4268b70f3936036e550b`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:04.749279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-08b31c86-544b-4a99-baba-2ecf874cd26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:04.809150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a84e5-6add-430d-b2fe-a4f98faf37e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:04.809890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a84e5-6add-430d-b2fe-a4f98faf37e6 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:04.810168 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a84e5-6add-430d-b2fe-a4f98faf37e6 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:04.810424 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a84e5-6add-430d-b2fe-a4f98faf37e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:04.812320 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-577a84e5-6add-430d-b2fe-a4f98faf37e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:04.830650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:04.860050 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 689/1401] 10.222.0.86 () {58 vars in 1040 bytes} [Fri Aug 5 19:43:04 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:43:04.870291 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b31c86-544b-4a99-baba-2ecf874cd26a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:04.880392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b31c86-544b-4a99-baba-2ecf874cd26a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:04.906285 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a84e5-6add-430d-b2fe-a4f98faf37e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:04.907918 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a84e5-6add-430d-b2fe-a4f98faf37e6 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:04.927125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 713/1402] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:43:04.998141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950c0d7-1f60-4372-a647-c366e95b3911 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:04.998801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950c0d7-1f60-4372-a647-c366e95b3911 service swift] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:04.999078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950c0d7-1f60-4372-a647-c366e95b3911 service swift] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:04.999348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950c0d7-1f60-4372-a647-c366e95b3911 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:05.001082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5950c0d7-1f60-4372-a647-c366e95b3911 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:05.004053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:05.097277 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5950c0d7-1f60-4372-a647-c366e95b3911 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:05.098915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5950c0d7-1f60-4372-a647-c366e95b3911 service swift] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:05.108479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 690/1403] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:43:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 19:43:07.032706 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748faef-7181-4432-ac82-c5a2f35fc870 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:07.033276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748faef-7181-4432-ac82-c5a2f35fc870 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:07.033276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748faef-7181-4432-ac82-c5a2f35fc870 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:07.034604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 714/1404] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:07.040612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8a1b8-00a4-4f8a-95e0-558f9077a5bc None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:07.041044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8a1b8-00a4-4f8a-95e0-558f9077a5bc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:07.041150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a8a1b8-00a4-4f8a-95e0-558f9077a5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:07.117142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-36a8a1b8-00a4-4f8a-95e0-558f9077a5bc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:07.118451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-36a8a1b8-00a4-4f8a-95e0-558f9077a5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:07.228346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:07.238676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 691/1405] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:07.291570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748faef-7181-4432-ac82-c5a2f35fc870 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:07.292980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8748faef-7181-4432-ac82-c5a2f35fc870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:07.509622 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:07.513279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748faef-7181-4432-ac82-c5a2f35fc870 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:07.653560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b555e76-ca94-4b6d-ada1-246a29c7eb4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:07.654095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b555e76-ca94-4b6d-ada1-246a29c7eb4b service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:07.654365 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b555e76-ca94-4b6d-ada1-246a29c7eb4b service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:07.654686 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b555e76-ca94-4b6d-ada1-246a29c7eb4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:07.656275 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5b555e76-ca94-4b6d-ada1-246a29c7eb4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:07.754788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b555e76-ca94-4b6d-ada1-246a29c7eb4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:07.756507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b555e76-ca94-4b6d-ada1-246a29c7eb4b service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:07.776369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 715/1406] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:09.934984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3b005-ab6a-4e6e-ae94-8946f5365d6a None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:09.935610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3b005-ab6a-4e6e-ae94-8946f5365d6a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:09.935610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3b005-ab6a-4e6e-ae94-8946f5365d6a None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:09.937318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 692/1407] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:09.942759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c3527-cef7-4284-beb7-12b03739d35b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:09.943214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c3527-cef7-4284-beb7-12b03739d35b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:09.943302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c3527-cef7-4284-beb7-12b03739d35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:09.947971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:10.024451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-633c3527-cef7-4284-beb7-12b03739d35b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:10.025766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-633c3527-cef7-4284-beb7-12b03739d35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:10.148565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 716/1408] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:10.193892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3b005-ab6a-4e6e-ae94-8946f5365d6a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:10.195402 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-75c3b005-ab6a-4e6e-ae94-8946f5365d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:10.371658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3b005-ab6a-4e6e-ae94-8946f5365d6a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:10.452724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:10.508562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b51c0-0b1e-4f96-85cf-dae7d85b6838 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:10.509185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b51c0-0b1e-4f96-85cf-dae7d85b6838 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:10.509462 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b51c0-0b1e-4f96-85cf-dae7d85b6838 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:10.509794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b51c0-0b1e-4f96-85cf-dae7d85b6838 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:10.512149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-628b51c0-0b1e-4f96-85cf-dae7d85b6838 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:10.614972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b51c0-0b1e-4f96-85cf-dae7d85b6838 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:10.616645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b51c0-0b1e-4f96-85cf-dae7d85b6838 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:10.640198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 693/1409] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:20.146213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=64 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:20.636628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=64 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:27.938181 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a3cd6-808f-4622-82cf-64f8549ab598 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:27.938793 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a3cd6-808f-4622-82cf-64f8549ab598 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:27.938793 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a3cd6-808f-4622-82cf-64f8549ab598 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:27.940109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 717/1410] 10.222.0.86 () {54 vars in 918 bytes} [Fri Aug 5 19:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:27.945161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe166c-dafd-4dc6-bbbe-df9cf9e7dd4c None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:27.945629 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe166c-dafd-4dc6-bbbe-df9cf9e7dd4c None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:27.945752 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe166c-dafd-4dc6-bbbe-df9cf9e7dd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:28.022219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-dfbe166c-dafd-4dc6-bbbe-df9cf9e7dd4c None None] MFA Rules not processed for user `cce71f93c80d433b830c448d0a7a14e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:28.023485 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-dfbe166c-dafd-4dc6-bbbe-df9cf9e7dd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:28.129764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 694/1411] 10.222.0.86 () {58 vars in 1015 bytes} [Fri Aug 5 19:43:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:28.144976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ffd29-c32b-420e-b1fc-fe6a40318895 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:28.145260 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ffd29-c32b-420e-b1fc-fe6a40318895 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:28.145535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ffd29-c32b-420e-b1fc-fe6a40318895 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:28.146943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 718/1412] 10.222.0.86 () {54 vars in 918 bytes} [Fri Aug 5 19:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:28.151788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cce0b-fca8-4e99-9e9f-d9c75b68480c None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:28.152074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cce0b-fca8-4e99-9e9f-d9c75b68480c None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:28.152349 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cce0b-fca8-4e99-9e9f-d9c75b68480c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:28.225946 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:28.231056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-538cce0b-fca8-4e99-9e9f-d9c75b68480c None None] MFA Rules not processed for user `ab8f9d9cf4ca4e38ae6eb67e949b40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:28.232902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-538cce0b-fca8-4e99-9e9f-d9c75b68480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:28.350206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 695/1413] 10.222.0.86 () {58 vars in 1015 bytes} [Fri Aug 5 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:43:30.870010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8737fe5-b768-44ca-ae6c-9588f7d7374e None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:30.870611 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8737fe5-b768-44ca-ae6c-9588f7d7374e None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:30.870754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8737fe5-b768-44ca-ae6c-9588f7d7374e None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:30.872124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:43:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:30.877729 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a3733-88ff-4ed5-ac8c-490d4663bb1b None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:30.878135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a3733-88ff-4ed5-ac8c-490d4663bb1b None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:30.878279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a3733-88ff-4ed5-ac8c-490d4663bb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:30.965076 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-d18a3733-88ff-4ed5-ac8c-490d4663bb1b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:30.966306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d18a3733-88ff-4ed5-ac8c-490d4663bb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:31.021537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:31.064720 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:31.107758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8737fe5-b768-44ca-ae6c-9588f7d7374e None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:31.109269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e8737fe5-b768-44ca-ae6c-9588f7d7374e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:31.260402 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8737fe5-b768-44ca-ae6c-9588f7d7374e None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:31.342411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:31.383932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c22e5-1455-4864-b938-28e18e728e65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:31.384562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c22e5-1455-4864-b938-28e18e728e65 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:31.384941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c22e5-1455-4864-b938-28e18e728e65 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:31.385213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c22e5-1455-4864-b938-28e18e728e65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:31.386865 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fb1c22e5-1455-4864-b938-28e18e728e65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:31.477319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c22e5-1455-4864-b938-28e18e728e65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:31.480189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c22e5-1455-4864-b938-28e18e728e65 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:31.499534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 720/1416] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:32.412801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc51d3-73f1-42ac-88ce-9d5fc687a67f None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:32.413377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc51d3-73f1-42ac-88ce-9d5fc687a67f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:32.413377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc51d3-73f1-42ac-88ce-9d5fc687a67f None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:32.414727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:32.420612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12925bc3-9da4-4df5-9e69-6752fc0ce2eb None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:32.421129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12925bc3-9da4-4df5-9e69-6752fc0ce2eb None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:32.421228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12925bc3-9da4-4df5-9e69-6752fc0ce2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:32.499273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-12925bc3-9da4-4df5-9e69-6752fc0ce2eb None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:32.500493 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-12925bc3-9da4-4df5-9e69-6752fc0ce2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:32.528158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:32.599906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:32.757107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc51d3-73f1-42ac-88ce-9d5fc687a67f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:32.758561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-8cfc51d3-73f1-42ac-88ce-9d5fc687a67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:32.905868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc51d3-73f1-42ac-88ce-9d5fc687a67f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:32.948555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:33.021897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0cb13-9d4d-4ba2-8bc0-b33f7c4e3525 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:33.022507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0cb13-9d4d-4ba2-8bc0-b33f7c4e3525 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:33.022813 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0cb13-9d4d-4ba2-8bc0-b33f7c4e3525 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:33.023085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0cb13-9d4d-4ba2-8bc0-b33f7c4e3525 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:33.024706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e0e0cb13-9d4d-4ba2-8bc0-b33f7c4e3525 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:33.115089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0cb13-9d4d-4ba2-8bc0-b33f7c4e3525 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:33.116688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0cb13-9d4d-4ba2-8bc0-b33f7c4e3525 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:33.135372 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 698/1419] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:34.030434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc3724-419f-46ad-8952-2c3f8b444341 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:34.030976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc3724-419f-46ad-8952-2c3f8b444341 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:34.031077 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc3724-419f-46ad-8952-2c3f8b444341 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:34.032378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:34.037273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10488f64-7d5b-4ee6-a750-65cfe856dfff None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:34.037682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10488f64-7d5b-4ee6-a750-65cfe856dfff None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:34.037803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10488f64-7d5b-4ee6-a750-65cfe856dfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:34.094888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:34.116772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-10488f64-7d5b-4ee6-a750-65cfe856dfff None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:34.117989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-10488f64-7d5b-4ee6-a750-65cfe856dfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:34.216031 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:34.370184 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc3724-419f-46ad-8952-2c3f8b444341 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:34.371775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-81dc3724-419f-46ad-8952-2c3f8b444341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:34.524527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc3724-419f-46ad-8952-2c3f8b444341 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:34.536976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:34.640413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbbe13-16e3-4645-b1eb-dea06a1c295c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:34.640935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbbe13-16e3-4645-b1eb-dea06a1c295c service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:34.641211 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbbe13-16e3-4645-b1eb-dea06a1c295c service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:34.641479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbbe13-16e3-4645-b1eb-dea06a1c295c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:34.643134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a0cbbe13-16e3-4645-b1eb-dea06a1c295c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:34.732977 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cbbe13-16e3-4645-b1eb-dea06a1c295c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:34.734570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cbbe13-16e3-4645-b1eb-dea06a1c295c service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:34.753100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 723/1422] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:35.632343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486681a-5d8b-4519-942b-f62f37cec015 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:35.632915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486681a-5d8b-4519-942b-f62f37cec015 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:35.632915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486681a-5d8b-4519-942b-f62f37cec015 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:35.634456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:35.639467 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4ba3e-22fc-4673-be0f-411e40e06789 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:35.639933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4ba3e-22fc-4673-be0f-411e40e06789 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:35.640042 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4ba3e-22fc-4673-be0f-411e40e06789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:35.660150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:35.719361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-1ea4ba3e-22fc-4673-be0f-411e40e06789 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:35.720581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1ea4ba3e-22fc-4673-be0f-411e40e06789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:35.818339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:35.973942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486681a-5d8b-4519-942b-f62f37cec015 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:35.975610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e486681a-5d8b-4519-942b-f62f37cec015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:36.086247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:36.123352 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486681a-5d8b-4519-942b-f62f37cec015 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:36.236512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafac8f-b470-40bd-b5e6-7dfc10470daf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:36.237131 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafac8f-b470-40bd-b5e6-7dfc10470daf service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:36.237404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafac8f-b470-40bd-b5e6-7dfc10470daf service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:36.237674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafac8f-b470-40bd-b5e6-7dfc10470daf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:36.239341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-8dafac8f-b470-40bd-b5e6-7dfc10470daf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:36.315991 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:36.330232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafac8f-b470-40bd-b5e6-7dfc10470daf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:36.331980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafac8f-b470-40bd-b5e6-7dfc10470daf service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:36.350397 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 701/1425] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:37.232581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b81d8f-a049-4955-b41e-f8df50563780 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:37.233159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b81d8f-a049-4955-b41e-f8df50563780 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:37.233159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b81d8f-a049-4955-b41e-f8df50563780 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:37.234486 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:37.239342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e998b-cbae-4ef2-8f5c-755e0aba801a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:37.239807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e998b-cbae-4ef2-8f5c-755e0aba801a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:37.239930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e998b-cbae-4ef2-8f5c-755e0aba801a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:37.316480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c66e998b-cbae-4ef2-8f5c-755e0aba801a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:37.317724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c66e998b-cbae-4ef2-8f5c-755e0aba801a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:37.415127 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:37.567507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b81d8f-a049-4955-b41e-f8df50563780 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:37.569016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e6b81d8f-a049-4955-b41e-f8df50563780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:37.652928 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:37.716698 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b81d8f-a049-4955-b41e-f8df50563780 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:37.828315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c4bfc-2133-4e43-983e-542d72750908 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:37.828834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c4bfc-2133-4e43-983e-542d72750908 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:37.829104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c4bfc-2133-4e43-983e-542d72750908 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:37.829369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c4bfc-2133-4e43-983e-542d72750908 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:37.831074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-128c4bfc-2133-4e43-983e-542d72750908 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:37.878643 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:37.919841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c4bfc-2133-4e43-983e-542d72750908 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:37.921435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c4bfc-2133-4e43-983e-542d72750908 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:37.939248 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 726/1428] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:38.754542 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237a4bb-9ebb-487f-ab4e-fb8433d16795 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:38.755175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237a4bb-9ebb-487f-ab4e-fb8433d16795 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:38.755175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237a4bb-9ebb-487f-ab4e-fb8433d16795 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:38.756394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:38.761361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722accd-4bf0-4548-a271-980e49853d41 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:38.761783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722accd-4bf0-4548-a271-980e49853d41 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:38.761888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722accd-4bf0-4548-a271-980e49853d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:38.841604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3722accd-4bf0-4548-a271-980e49853d41 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:38.842835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3722accd-4bf0-4548-a271-980e49853d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:38.939875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:43:39.091309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237a4bb-9ebb-487f-ab4e-fb8433d16795 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:39.092816 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-4237a4bb-9ebb-487f-ab4e-fb8433d16795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:39.141765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:39.242885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237a4bb-9ebb-487f-ab4e-fb8433d16795 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:39.357442 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aabe78-c947-439c-a742-c39a2687c82d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:39.357968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aabe78-c947-439c-a742-c39a2687c82d service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:39.358245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aabe78-c947-439c-a742-c39a2687c82d service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:39.358512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aabe78-c947-439c-a742-c39a2687c82d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:39.360177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-73aabe78-c947-439c-a742-c39a2687c82d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:39.367083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:39.458908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aabe78-c947-439c-a742-c39a2687c82d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:39.460520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aabe78-c947-439c-a742-c39a2687c82d service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:39.479528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 704/1431] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:40.331713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244df404-d634-445b-873c-be18a1665844 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:40.332324 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244df404-d634-445b-873c-be18a1665844 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:40.332438 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244df404-d634-445b-873c-be18a1665844 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:40.333825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:40.339342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf36e3-64d8-49fa-855b-aab588549559 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:40.339771 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf36e3-64d8-49fa-855b-aab588549559 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:40.339871 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf36e3-64d8-49fa-855b-aab588549559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:40.415009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-e4bf36e3-64d8-49fa-855b-aab588549559 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:40.416361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e4bf36e3-64d8-49fa-855b-aab588549559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:40.484993 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:40.519551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:43:40.670503 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244df404-d634-445b-873c-be18a1665844 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:40.671966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-244df404-d634-445b-873c-be18a1665844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:40.690358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:40.820680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244df404-d634-445b-873c-be18a1665844 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:40.907412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:40.934484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796ad63-b85f-41ef-b94e-8368cb249c0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:40.935124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796ad63-b85f-41ef-b94e-8368cb249c0d service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:40.935651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796ad63-b85f-41ef-b94e-8368cb249c0d service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:40.935922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796ad63-b85f-41ef-b94e-8368cb249c0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:40.937549 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2796ad63-b85f-41ef-b94e-8368cb249c0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:41.031583 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796ad63-b85f-41ef-b94e-8368cb249c0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:41.033181 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796ad63-b85f-41ef-b94e-8368cb249c0d service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:41.051591 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 729/1434] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:41.859650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a3ea8-087c-4e3c-80e5-90efcb786e1e None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:41.860514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a3ea8-087c-4e3c-80e5-90efcb786e1e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:41.860514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a3ea8-087c-4e3c-80e5-90efcb786e1e None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:41.862201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:43:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:41.867082 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7fbee-d12e-4102-b720-be13fb22be2d None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:41.867503 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7fbee-d12e-4102-b720-be13fb22be2d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:41.867619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7fbee-d12e-4102-b720-be13fb22be2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:41.942797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-64b7fbee-d12e-4102-b720-be13fb22be2d None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:41.943996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-64b7fbee-d12e-4102-b720-be13fb22be2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:41.987482 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:42.043640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:42.197702 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a3ea8-087c-4e3c-80e5-90efcb786e1e None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:42.199158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6e5a3ea8-087c-4e3c-80e5-90efcb786e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:42.347540 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a3ea8-087c-4e3c-80e5-90efcb786e1e None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:42.402741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:42.463283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1971886-6a2b-4738-a9af-53fb811cdfff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:42.463940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1971886-6a2b-4738-a9af-53fb811cdfff service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:42.464240 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1971886-6a2b-4738-a9af-53fb811cdfff service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:42.464508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1971886-6a2b-4738-a9af-53fb811cdfff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:42.466128 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d1971886-6a2b-4738-a9af-53fb811cdfff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:42.561369 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1971886-6a2b-4738-a9af-53fb811cdfff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:42.563036 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1971886-6a2b-4738-a9af-53fb811cdfff service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:42.582364 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 707/1437] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:43.446605 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d2d7a-e321-4463-851a-a7d3c24e2a33 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:43.447393 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d2d7a-e321-4463-851a-a7d3c24e2a33 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:43.447393 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d2d7a-e321-4463-851a-a7d3c24e2a33 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:43.448803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:43.454406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68a456-df17-4f3e-b9e0-ee0d248ee9f3 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:43.454950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68a456-df17-4f3e-b9e0-ee0d248ee9f3 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:43.454950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68a456-df17-4f3e-b9e0-ee0d248ee9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:43.539082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-de68a456-df17-4f3e-b9e0-ee0d248ee9f3 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:43.541500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-de68a456-df17-4f3e-b9e0-ee0d248ee9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:43.544340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:43.642570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:43.800124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d2d7a-e321-4463-851a-a7d3c24e2a33 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:43.801722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e51d2d7a-e321-4463-851a-a7d3c24e2a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:43.952448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d2d7a-e321-4463-851a-a7d3c24e2a33 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:43.979774 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:44.067886 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f24d9-8c52-4dd6-839f-3accca52e1ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:44.068498 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f24d9-8c52-4dd6-839f-3accca52e1ab service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:44.068773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f24d9-8c52-4dd6-839f-3accca52e1ab service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:44.069041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f24d9-8c52-4dd6-839f-3accca52e1ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:44.070690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e08f24d9-8c52-4dd6-839f-3accca52e1ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:44.161250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f24d9-8c52-4dd6-839f-3accca52e1ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:44.162942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f24d9-8c52-4dd6-839f-3accca52e1ab service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:44.181534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 732/1440] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:46.110974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0d5ef-3db5-4b65-8fb0-38a8471fee5c None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:46.111512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0d5ef-3db5-4b65-8fb0-38a8471fee5c None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:46.111512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0d5ef-3db5-4b65-8fb0-38a8471fee5c None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:46.112850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:46.118093 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6134ad-5f8d-49da-bea2-7d9e72b14dcc None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:46.119377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6134ad-5f8d-49da-bea2-7d9e72b14dcc None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:46.119377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6134ad-5f8d-49da-bea2-7d9e72b14dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:46.161125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:46.187574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-5f6134ad-5f8d-49da-bea2-7d9e72b14dcc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:46.188767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5f6134ad-5f8d-49da-bea2-7d9e72b14dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:46.284194 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:46.324367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0d5ef-3db5-4b65-8fb0-38a8471fee5c None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:46.325888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-01b0d5ef-3db5-4b65-8fb0-38a8471fee5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:46.431372 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0d5ef-3db5-4b65-8fb0-38a8471fee5c None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:46.464301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:46.508976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ae5bb-15db-4359-8b39-d22510c05fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:46.509735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ae5bb-15db-4359-8b39-d22510c05fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:46.509850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ae5bb-15db-4359-8b39-d22510c05fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:46.510105 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ae5bb-15db-4359-8b39-d22510c05fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:46.512176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-999ae5bb-15db-4359-8b39-d22510c05fe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:46.644880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ae5bb-15db-4359-8b39-d22510c05fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:46.646221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ae5bb-15db-4359-8b39-d22510c05fe0 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:46.664519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 710/1443] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:48.872685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fa5e3-dea7-4614-96ea-9f246d7d7666 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:48.873258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fa5e3-dea7-4614-96ea-9f246d7d7666 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:48.873258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fa5e3-dea7-4614-96ea-9f246d7d7666 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:48.874652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:48.879976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f8ac0-f678-489f-8766-6ccbaeeceedd None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:48.880411 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f8ac0-f678-489f-8766-6ccbaeeceedd None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:48.880521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f8ac0-f678-489f-8766-6ccbaeeceedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:48.948900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-421f8ac0-f678-489f-8766-6ccbaeeceedd None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:48.950098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-421f8ac0-f678-489f-8766-6ccbaeeceedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:48.963381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:49.048287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:49.087062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589fa5e3-dea7-4614-96ea-9f246d7d7666 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:49.088662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-589fa5e3-dea7-4614-96ea-9f246d7d7666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:49.193699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589fa5e3-dea7-4614-96ea-9f246d7d7666 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:49.206022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:49.268515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3ad84-46a2-4a5f-8f6a-43d80f22ef08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:49.269150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3ad84-46a2-4a5f-8f6a-43d80f22ef08 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:49.269422 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3ad84-46a2-4a5f-8f6a-43d80f22ef08 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:49.269690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3ad84-46a2-4a5f-8f6a-43d80f22ef08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:49.271338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f5b3ad84-46a2-4a5f-8f6a-43d80f22ef08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:49.380384 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3ad84-46a2-4a5f-8f6a-43d80f22ef08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:49.381804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3ad84-46a2-4a5f-8f6a-43d80f22ef08 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:49.401616 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 735/1446] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:51.777467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5f809-774a-496f-968e-b9f552a9287d None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:51.778040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5f809-774a-496f-968e-b9f552a9287d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:51.778040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5f809-774a-496f-968e-b9f552a9287d None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:51.779390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:51.784803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e224a-491b-4445-a59a-86562fc352b5 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:51.785222 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e224a-491b-4445-a59a-86562fc352b5 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:51.785374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e224a-491b-4445-a59a-86562fc352b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:51.833041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:51.851151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-f99e224a-491b-4445-a59a-86562fc352b5 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:51.853003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f99e224a-491b-4445-a59a-86562fc352b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:51.949479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:51.988743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5f809-774a-496f-968e-b9f552a9287d None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:51.990263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c6e5f809-774a-496f-968e-b9f552a9287d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:52.095610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5f809-774a-496f-968e-b9f552a9287d None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:52.136263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:52.171417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8c88d-0530-4dd1-b447-2fc5b3aef64a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:52.172033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8c88d-0530-4dd1-b447-2fc5b3aef64a service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:52.172311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8c88d-0530-4dd1-b447-2fc5b3aef64a service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:52.172583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8c88d-0530-4dd1-b447-2fc5b3aef64a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:52.174277 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-79a8c88d-0530-4dd1-b447-2fc5b3aef64a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:52.283963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8c88d-0530-4dd1-b447-2fc5b3aef64a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:52.285394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8c88d-0530-4dd1-b447-2fc5b3aef64a service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:52.304373 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 713/1449] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:54.494159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28387194-fb2a-479d-8c74-86da54c25a93 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:54.494673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28387194-fb2a-479d-8c74-86da54c25a93 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:54.494783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28387194-fb2a-479d-8c74-86da54c25a93 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:54.496261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:54.501680 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7b46e-914e-4c3b-8f09-19d991524170 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:54.502088 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7b46e-914e-4c3b-8f09-19d991524170 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:54.502271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7b46e-914e-4c3b-8f09-19d991524170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:54.570192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-3fc7b46e-914e-4c3b-8f09-19d991524170 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:54.571597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3fc7b46e-914e-4c3b-8f09-19d991524170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:54.593521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:54.669860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:54.710473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28387194-fb2a-479d-8c74-86da54c25a93 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:54.712666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-28387194-fb2a-479d-8c74-86da54c25a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:54.815536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28387194-fb2a-479d-8c74-86da54c25a93 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:54.835464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:54.889048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97056d1-2679-4049-8383-953b02aefe03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:54.889660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97056d1-2679-4049-8383-953b02aefe03 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:54.889932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97056d1-2679-4049-8383-953b02aefe03 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:54.890201 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97056d1-2679-4049-8383-953b02aefe03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:54.891845 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c97056d1-2679-4049-8383-953b02aefe03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:55.000353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97056d1-2679-4049-8383-953b02aefe03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:55.001790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97056d1-2679-4049-8383-953b02aefe03 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:55.020501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 738/1452] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:57.316562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dbd5c-f892-4052-8745-b19fb1692ac3 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:57.317113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dbd5c-f892-4052-8745-b19fb1692ac3 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:57.317113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dbd5c-f892-4052-8745-b19fb1692ac3 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:57.318439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:57.325003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f4bd2-f035-4739-b61b-d3d5e4098b86 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:57.325472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f4bd2-f035-4739-b61b-d3d5e4098b86 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:57.325573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f4bd2-f035-4739-b61b-d3d5e4098b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:57.389709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:57.392849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-b11f4bd2-f035-4739-b61b-d3d5e4098b86 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:43:57.394487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b11f4bd2-f035-4739-b61b-d3d5e4098b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:57.492590 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:43:57.531808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dbd5c-f892-4052-8745-b19fb1692ac3 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:43:57.533341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a65dbd5c-f892-4052-8745-b19fb1692ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:57.637999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dbd5c-f892-4052-8745-b19fb1692ac3 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:43:57.688148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:43:57.713674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff6e51-5622-4cf9-8e99-adab6f84725d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:43:57.714287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6e51-5622-4cf9-8e99-adab6f84725d service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:57.714563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6e51-5622-4cf9-8e99-adab6f84725d service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:57.714917 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6e51-5622-4cf9-8e99-adab6f84725d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:57.716493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-11ff6e51-5622-4cf9-8e99-adab6f84725d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:43:57.825882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff6e51-5622-4cf9-8e99-adab6f84725d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:43:57.827357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff6e51-5622-4cf9-8e99-adab6f84725d service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:43:57.845524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 716/1455] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:43:59.941990 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b4fdb-d4a0-4361-9599-f1d37874b09b None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:59.942573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b4fdb-d4a0-4361-9599-f1d37874b09b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:59.942573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b4fdb-d4a0-4361-9599-f1d37874b09b None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:43:59.943934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:43:59.949998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e40842-c7cd-40ce-bdd9-3a3a0298f6cc None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:43:59.950463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e40842-c7cd-40ce-bdd9-3a3a0298f6cc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:43:59.950564 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e40842-c7cd-40ce-bdd9-3a3a0298f6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:00.016823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-33e40842-c7cd-40ce-bdd9-3a3a0298f6cc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:44:00.018017 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-33e40842-c7cd-40ce-bdd9-3a3a0298f6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:00.049673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:00.238463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:44:00.279769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b4fdb-d4a0-4361-9599-f1d37874b09b None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:44:00.281299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-996b4fdb-d4a0-4361-9599-f1d37874b09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:00.385796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b4fdb-d4a0-4361-9599-f1d37874b09b None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:44:00.413294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:00.459683 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee732a-c159-4f31-8f72-b0078e67b076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:44:00.460297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee732a-c159-4f31-8f72-b0078e67b076 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:00.460574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee732a-c159-4f31-8f72-b0078e67b076 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:00.460840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee732a-c159-4f31-8f72-b0078e67b076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:00.462704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-dfee732a-c159-4f31-8f72-b0078e67b076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:00.573710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee732a-c159-4f31-8f72-b0078e67b076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:44:00.575185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee732a-c159-4f31-8f72-b0078e67b076 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:44:00.593633 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 741/1458] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:44:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:44:02.674981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131305d-9ac3-41df-ac77-9ef6eb665997 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:02.675561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131305d-9ac3-41df-ac77-9ef6eb665997 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:02.675561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131305d-9ac3-41df-ac77-9ef6eb665997 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:02.676861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:44:02.682208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4df21b-b8e4-4081-bc56-0aa65ca106e0 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:02.682724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4df21b-b8e4-4081-bc56-0aa65ca106e0 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:02.682831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4df21b-b8e4-4081-bc56-0aa65ca106e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:02.749204 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-4b4df21b-b8e4-4081-bc56-0aa65ca106e0 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:44:02.750330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4b4df21b-b8e4-4081-bc56-0aa65ca106e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:02.758819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:02.855043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:44:02.892840 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131305d-9ac3-41df-ac77-9ef6eb665997 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:44:02.894361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6131305d-9ac3-41df-ac77-9ef6eb665997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:02.996613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131305d-9ac3-41df-ac77-9ef6eb665997 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:44:03.051239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:03.069390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee829732-e88c-4871-8e01-f1ecb3767acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:44:03.069996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee829732-e88c-4871-8e01-f1ecb3767acb service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:03.070267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee829732-e88c-4871-8e01-f1ecb3767acb service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:03.070533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee829732-e88c-4871-8e01-f1ecb3767acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:03.072789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ee829732-e88c-4871-8e01-f1ecb3767acb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:03.184157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee829732-e88c-4871-8e01-f1ecb3767acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:44:03.185835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee829732-e88c-4871-8e01-f1ecb3767acb service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:44:03.204146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 719/1461] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:44:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:44:05.316879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a4a84-a09b-4aa0-8d5b-da30fad1441a None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:05.317425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a4a84-a09b-4aa0-8d5b-da30fad1441a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:05.317425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a4a84-a09b-4aa0-8d5b-da30fad1441a None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:05.318871 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:44:05.325534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c474e2-e822-4f8c-8457-c25209ed77e8 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:05.326005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c474e2-e822-4f8c-8457-c25209ed77e8 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:05.326104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c474e2-e822-4f8c-8457-c25209ed77e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:05.391209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-96c474e2-e822-4f8c-8457-c25209ed77e8 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:44:05.392422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-96c474e2-e822-4f8c-8457-c25209ed77e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:05.432335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:05.491972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:44:05.530744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a4a84-a09b-4aa0-8d5b-da30fad1441a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:44:05.532266 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-829a4a84-a09b-4aa0-8d5b-da30fad1441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:05.639507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a4a84-a09b-4aa0-8d5b-da30fad1441a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:44:05.677859 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:05.715623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3aae64-81e9-4afb-a669-3f0fd7670d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:44:05.716249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3aae64-81e9-4afb-a669-3f0fd7670d17 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:05.716529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3aae64-81e9-4afb-a669-3f0fd7670d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:05.716802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3aae64-81e9-4afb-a669-3f0fd7670d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:05.718446 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fc3aae64-81e9-4afb-a669-3f0fd7670d17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:05.828853 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3aae64-81e9-4afb-a669-3f0fd7670d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:44:05.830286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3aae64-81e9-4afb-a669-3f0fd7670d17 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:44:05.850076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 744/1464] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:44:07.887923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfbb9a-86e2-42ae-bdfb-7e0128107dbf None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:07.888513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfbb9a-86e2-42ae-bdfb-7e0128107dbf None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:07.888513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfbb9a-86e2-42ae-bdfb-7e0128107dbf None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:07.889931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:44:07.896051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf981cfd-39cc-4367-98a5-9b1caf65c32a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:07.896513 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf981cfd-39cc-4367-98a5-9b1caf65c32a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:07.896625 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf981cfd-39cc-4367-98a5-9b1caf65c32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:07.964571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-cf981cfd-39cc-4367-98a5-9b1caf65c32a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:44:07.965785 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-cf981cfd-39cc-4367-98a5-9b1caf65c32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:07.984377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:08.065409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:44:08.102774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfbb9a-86e2-42ae-bdfb-7e0128107dbf None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:44:08.104359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b8bfbb9a-86e2-42ae-bdfb-7e0128107dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:08.208425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfbb9a-86e2-42ae-bdfb-7e0128107dbf None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:44:08.277808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:08.284996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd0345-611e-4568-9dfe-412fee9ac46b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:44:08.285603 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd0345-611e-4568-9dfe-412fee9ac46b service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:08.285875 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd0345-611e-4568-9dfe-412fee9ac46b service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:08.286140 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd0345-611e-4568-9dfe-412fee9ac46b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:08.287771 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c0bd0345-611e-4568-9dfe-412fee9ac46b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:08.400062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd0345-611e-4568-9dfe-412fee9ac46b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:44:08.401391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd0345-611e-4568-9dfe-412fee9ac46b service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:44:08.420318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 722/1467] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:44:10.468781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a1a7e-a52c-4428-9db6-8000978a799b None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:10.469369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a1a7e-a52c-4428-9db6-8000978a799b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:10.469369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a1a7e-a52c-4428-9db6-8000978a799b None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:10.470670 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:44:10.476048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e116c-a373-40c8-bdb0-041de49b6e6e None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:10.476481 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e116c-a373-40c8-bdb0-041de49b6e6e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:10.477069 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e116c-a373-40c8-bdb0-041de49b6e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:10.542843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-897e116c-a373-40c8-bdb0-041de49b6e6e None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:44:10.544054 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-897e116c-a373-40c8-bdb0-041de49b6e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:10.592620 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:10.642458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:44:10.680832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a1a7e-a52c-4428-9db6-8000978a799b None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:44:10.682354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-924a1a7e-a52c-4428-9db6-8000978a799b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:10.786080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a1a7e-a52c-4428-9db6-8000978a799b None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:44:10.834468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:10.862858 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12dd54-91c7-4ad9-bbe9-a1cc58b677c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:44:10.863470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12dd54-91c7-4ad9-bbe9-a1cc58b677c6 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:10.863740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12dd54-91c7-4ad9-bbe9-a1cc58b677c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:10.864003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12dd54-91c7-4ad9-bbe9-a1cc58b677c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:10.865601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5d12dd54-91c7-4ad9-bbe9-a1cc58b677c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:10.977948 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12dd54-91c7-4ad9-bbe9-a1cc58b677c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:44:10.979645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12dd54-91c7-4ad9-bbe9-a1cc58b677c6 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:44:11.003634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 747/1470] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:44:20.640591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=22 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:21.000647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=66 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:30.889438 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e9b0b-0d8f-4aa5-9b30-4bf421b8ab96 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:30.890054 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e9b0b-0d8f-4aa5-9b30-4bf421b8ab96 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:30.890054 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e9b0b-0d8f-4aa5-9b30-4bf421b8ab96 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:30.891378 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 724/1471] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:44:30.896758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6468d1-cb3c-44ca-a71a-e73622bcb30b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:30.897157 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6468d1-cb3c-44ca-a71a-e73622bcb30b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:30.897302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6468d1-cb3c-44ca-a71a-e73622bcb30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:30.966706 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-9f6468d1-cb3c-44ca-a71a-e73622bcb30b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:44:30.967973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9f6468d1-cb3c-44ca-a71a-e73622bcb30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:31.067120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 748/1472] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:44:31.151207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1beb872-130d-483a-83fa-00cb2c7a4a3f None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:31.151487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1beb872-130d-483a-83fa-00cb2c7a4a3f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:31.151755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1beb872-130d-483a-83fa-00cb2c7a4a3f None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:31.153082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 725/1473] 10.222.0.86 () {54 vars in 966 bytes} [Fri Aug 5 19:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:44:31.157922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ef122-8ac3-440f-81d5-d4284031f424 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:31.158209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ef122-8ac3-440f-81d5-d4284031f424 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:31.158484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ef122-8ac3-440f-81d5-d4284031f424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:31.237223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-d62ef122-8ac3-440f-81d5-d4284031f424 None None] MFA Rules not processed for user `72d4459164844e718544d7fde333db76`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:44:31.238430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d62ef122-8ac3-440f-81d5-d4284031f424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:31.259618 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:31.317566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 749/1474] 10.222.0.86 () {58 vars in 1063 bytes} [Fri Aug 5 19:44:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:44:31.325223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1beb872-130d-483a-83fa-00cb2c7a4a3f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:44:31.326790 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e1beb872-130d-483a-83fa-00cb2c7a4a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:31.431923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1beb872-130d-483a-83fa-00cb2c7a4a3f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:44:31.509046 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8edd1f-2b21-4fc7-96e9-6843c7a83d42 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72d4459164844e718544d7fde333db76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:44:31.509748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8edd1f-2b21-4fc7-96e9-6843c7a83d42 service glance] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:31.510022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8edd1f-2b21-4fc7-96e9-6843c7a83d42 service glance] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:31.510293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8edd1f-2b21-4fc7-96e9-6843c7a83d42 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:31.511954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-1e8edd1f-2b21-4fc7-96e9-6843c7a83d42 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:31.566010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:31.622293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8edd1f-2b21-4fc7-96e9-6843c7a83d42 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:44:31.623746 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8edd1f-2b21-4fc7-96e9-6843c7a83d42 service glance] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:44:31.643192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 726/1475] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:44:33.774554 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd179443-ea13-459d-a5ee-c4b4f7f6c69f None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:33.775135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd179443-ea13-459d-a5ee-c4b4f7f6c69f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:33.775254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd179443-ea13-459d-a5ee-c4b4f7f6c69f None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:33.776576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 750/1476] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:44:33.781900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0b11a-13df-4c7a-9a74-8fb1cd2dc358 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:33.782306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0b11a-13df-4c7a-9a74-8fb1cd2dc358 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:33.782455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0b11a-13df-4c7a-9a74-8fb1cd2dc358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:33.848985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-81e0b11a-13df-4c7a-9a74-8fb1cd2dc358 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:44:33.850238 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-81e0b11a-13df-4c7a-9a74-8fb1cd2dc358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:33.948428 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 727/1477] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:44:33.971330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd179443-ea13-459d-a5ee-c4b4f7f6c69f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:44:33.972853 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-dd179443-ea13-459d-a5ee-c4b4f7f6c69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:34.101624 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd179443-ea13-459d-a5ee-c4b4f7f6c69f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:44:34.149912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:34.196930 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f83cf-a5da-4a8d-8d03-d0aed5c0d7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:44:34.197535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f83cf-a5da-4a8d-8d03-d0aed5c0d7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:34.197801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f83cf-a5da-4a8d-8d03-d0aed5c0d7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:34.198068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f83cf-a5da-4a8d-8d03-d0aed5c0d7d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:34.199008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f83cf-a5da-4a8d-8d03-d0aed5c0d7d0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:44:34.200673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: WARNING py.warnings [None req-cc4f83cf-a5da-4a8d-8d03-d0aed5c0d7d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:44:34.200673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: warnings.warn(msg) Aug 05 19:44:34.200673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: Aug 05 19:44:34.201873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f83cf-a5da-4a8d-8d03-d0aed5c0d7d0 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:44:34.221301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 751/1478] 10.222.0.86 () {56 vars in 1108 bytes} [Fri Aug 5 19:44:33 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:44:36.294619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dae2b-7347-46fe-b2c1-fba55e78d7e5 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:36.295192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dae2b-7347-46fe-b2c1-fba55e78d7e5 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:36.295192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dae2b-7347-46fe-b2c1-fba55e78d7e5 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:36.296482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 728/1479] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:44:36.301767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb4266-5f5b-4297-931a-c9febd8dd0b8 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:36.302214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb4266-5f5b-4297-931a-c9febd8dd0b8 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:36.302330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb4266-5f5b-4297-931a-c9febd8dd0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:36.369449 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-7bbb4266-5f5b-4297-931a-c9febd8dd0b8 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:44:36.370689 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7bbb4266-5f5b-4297-931a-c9febd8dd0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:36.468230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 752/1480] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:44:36.511978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dae2b-7347-46fe-b2c1-fba55e78d7e5 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:44:36.513516 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-188dae2b-7347-46fe-b2c1-fba55e78d7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:36.524152 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:36.662714 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dae2b-7347-46fe-b2c1-fba55e78d7e5 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:44:36.744276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:36.778849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6317f-855f-43d4-9cdf-651a6cb9dff4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:44:36.779758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6317f-855f-43d4-9cdf-651a6cb9dff4 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:36.780045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6317f-855f-43d4-9cdf-651a6cb9dff4 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:36.780763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6317f-855f-43d4-9cdf-651a6cb9dff4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:36.782205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b4a6317f-855f-43d4-9cdf-651a6cb9dff4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:36.873873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6317f-855f-43d4-9cdf-651a6cb9dff4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:44:36.875514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6317f-855f-43d4-9cdf-651a6cb9dff4 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:44:36.894233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 729/1481] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:44:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:44:39.122124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a3dad-0095-4999-9b31-7f7f698cba5e None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:39.122743 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a3dad-0095-4999-9b31-7f7f698cba5e None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:39.122743 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a3dad-0095-4999-9b31-7f7f698cba5e None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:39.124107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 753/1482] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:44:39.129235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8ae6d-54b5-464a-a3a4-8fd6367139bc None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:39.129666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8ae6d-54b5-464a-a3a4-8fd6367139bc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:39.129795 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8ae6d-54b5-464a-a3a4-8fd6367139bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:39.198899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c8f8ae6d-54b5-464a-a3a4-8fd6367139bc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:44:39.200134 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c8f8ae6d-54b5-464a-a3a4-8fd6367139bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:39.246578 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:39.298025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 730/1483] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:44:39.336224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a3dad-0095-4999-9b31-7f7f698cba5e None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:44:39.338087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-724a3dad-0095-4999-9b31-7f7f698cba5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:39.361452 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:39.446165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a3dad-0095-4999-9b31-7f7f698cba5e None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:44:39.522735 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2a7b6-c6f4-4b87-9a85-6eab6907f9e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:44:39.523641 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2a7b6-c6f4-4b87-9a85-6eab6907f9e8 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:39.523774 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2a7b6-c6f4-4b87-9a85-6eab6907f9e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:39.524047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2a7b6-c6f4-4b87-9a85-6eab6907f9e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:39.525650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fae2a7b6-c6f4-4b87-9a85-6eab6907f9e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:39.608410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:39.644709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2a7b6-c6f4-4b87-9a85-6eab6907f9e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:44:39.646146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2a7b6-c6f4-4b87-9a85-6eab6907f9e8 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:44:39.669121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 754/1484] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:44:41.754807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bb4b5-bdbc-48c5-b0dd-bc35324bd208 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:41.755422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bb4b5-bdbc-48c5-b0dd-bc35324bd208 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:41.755422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bb4b5-bdbc-48c5-b0dd-bc35324bd208 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:41.757335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 731/1485] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:44:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:44:41.762798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5528ef-ab83-4ea4-b582-57f602db7150 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:41.763251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5528ef-ab83-4ea4-b582-57f602db7150 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:41.763356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5528ef-ab83-4ea4-b582-57f602db7150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:41.828826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-7c5528ef-ab83-4ea4-b582-57f602db7150 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:44:41.830029 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7c5528ef-ab83-4ea4-b582-57f602db7150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:41.929429 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 755/1486] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:44:41.967711 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bb4b5-bdbc-48c5-b0dd-bc35324bd208 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:44:41.969232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-342bb4b5-bdbc-48c5-b0dd-bc35324bd208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:42.074551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bb4b5-bdbc-48c5-b0dd-bc35324bd208 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:44:42.145198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:42.147287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283863b-3b78-42d2-b031-ad44982ec67d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:44:42.148172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283863b-3b78-42d2-b031-ad44982ec67d service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:42.148590 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283863b-3b78-42d2-b031-ad44982ec67d service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:42.149030 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283863b-3b78-42d2-b031-ad44982ec67d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:42.150858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d283863b-3b78-42d2-b031-ad44982ec67d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:42.265551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283863b-3b78-42d2-b031-ad44982ec67d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:44:42.267015 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283863b-3b78-42d2-b031-ad44982ec67d service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:44:42.284768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 732/1487] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:44:44.491093 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7022fd-806a-445c-a776-88a8c2915c2c None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:44.492251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7022fd-806a-445c-a776-88a8c2915c2c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:44.492251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7022fd-806a-445c-a776-88a8c2915c2c None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:44.494018 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 756/1488] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:44:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:44:44.499340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d088935-936c-4b9b-beda-8d2338bf04f4 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:44.499761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d088935-936c-4b9b-beda-8d2338bf04f4 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:44.499875 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d088935-936c-4b9b-beda-8d2338bf04f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:44.566048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-6d088935-936c-4b9b-beda-8d2338bf04f4 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:44:44.567301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6d088935-936c-4b9b-beda-8d2338bf04f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:44.621119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:44.666141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 733/1489] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:44:44.707403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7022fd-806a-445c-a776-88a8c2915c2c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:44:44.708931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4a7022fd-806a-445c-a776-88a8c2915c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:44.745962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:44.819816 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7022fd-806a-445c-a776-88a8c2915c2c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:44:44.892064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e0b1b-5b56-4738-b84c-ea420a1de001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:44:44.892665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e0b1b-5b56-4738-b84c-ea420a1de001 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:44.892936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e0b1b-5b56-4738-b84c-ea420a1de001 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:44.893203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e0b1b-5b56-4738-b84c-ea420a1de001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:44.894826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-624e0b1b-5b56-4738-b84c-ea420a1de001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:44.974418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:45.003032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e0b1b-5b56-4738-b84c-ea420a1de001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:44:45.004404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e0b1b-5b56-4738-b84c-ea420a1de001 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:44:45.023585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 757/1490] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:44:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:44:47.264284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a40eab-872e-4d18-b597-6b9598bf797f None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:47.264837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a40eab-872e-4d18-b597-6b9598bf797f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:47.264837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a40eab-872e-4d18-b597-6b9598bf797f None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:47.266191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 734/1491] 10.222.0.86 () {54 vars in 927 bytes} [Fri Aug 5 19:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:44:47.271487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb57362-ceff-4d7e-adce-2648193aa6cf None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:47.271886 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb57362-ceff-4d7e-adce-2648193aa6cf None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:47.272022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb57362-ceff-4d7e-adce-2648193aa6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:47.341651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-6fb57362-ceff-4d7e-adce-2648193aa6cf None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:44:47.342898 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6fb57362-ceff-4d7e-adce-2648193aa6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:47.439329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 758/1492] 10.222.0.86 () {58 vars in 1024 bytes} [Fri Aug 5 19:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:44:47.479379 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a40eab-872e-4d18-b597-6b9598bf797f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:44:47.480964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-76a40eab-872e-4d18-b597-6b9598bf797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:47.585185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a40eab-872e-4d18-b597-6b9598bf797f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:44:47.658791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19d03a-50b3-47cf-8c49-b188d3d8ab53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:44:47.659403 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19d03a-50b3-47cf-8c49-b188d3d8ab53 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:44:47.659678 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19d03a-50b3-47cf-8c49-b188d3d8ab53 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:44:47.659945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19d03a-50b3-47cf-8c49-b188d3d8ab53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:44:47.661551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-8f19d03a-50b3-47cf-8c49-b188d3d8ab53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:44:47.671761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:47.775202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19d03a-50b3-47cf-8c49-b188d3d8ab53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:44:47.776642 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19d03a-50b3-47cf-8c49-b188d3d8ab53 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:44:47.796587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 735/1493] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:44:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:44:57.437454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=82 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:44:57.794279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=50 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:45:59.695872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f23c9f-bd5a-41df-a6cc-fe3e3372fb7a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:45:59.696474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f23c9f-bd5a-41df-a6cc-fe3e3372fb7a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:45:59.696474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f23c9f-bd5a-41df-a6cc-fe3e3372fb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:45:59.773411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-a8f23c9f-bd5a-41df-a6cc-fe3e3372fb7a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:45:59.774601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a8f23c9f-bd5a-41df-a6cc-fe3e3372fb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:45:59.870411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 759/1494] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:45:59.879330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280cc92-f432-45bd-ad3b-29d594831748 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:45:59.879862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280cc92-f432-45bd-ad3b-29d594831748 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:45:59.880037 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280cc92-f432-45bd-ad3b-29d594831748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:45:59.957855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c280cc92-f432-45bd-ad3b-29d594831748 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:45:59.959108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c280cc92-f432-45bd-ad3b-29d594831748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:00.058075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 736/1495] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:46:00.063150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f23c9f-bd5a-41df-a6cc-fe3e3372fb7a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:46:00.064358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a8f23c9f-bd5a-41df-a6cc-fe3e3372fb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:00.076381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f23c9f-bd5a-41df-a6cc-fe3e3372fb7a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:46:00.139895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:46:00.174334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc81bb3-c9fe-4ba2-9ba5-4aa9d22110fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:46:00.174971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc81bb3-c9fe-4ba2-9ba5-4aa9d22110fc admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:00.175249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc81bb3-c9fe-4ba2-9ba5-4aa9d22110fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:00.176530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc81bb3-c9fe-4ba2-9ba5-4aa9d22110fc admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:00.177229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc81bb3-c9fe-4ba2-9ba5-4aa9d22110fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:46:00.177526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc81bb3-c9fe-4ba2-9ba5-4aa9d22110fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:46:00.178529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: WARNING py.warnings [None req-dcc81bb3-c9fe-4ba2-9ba5-4aa9d22110fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:46:00.178529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: warnings.warn(msg) Aug 05 19:46:00.178529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: Aug 05 19:46:00.179425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc81bb3-c9fe-4ba2-9ba5-4aa9d22110fc admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:46:00.190606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 760/1496] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:46:00.200025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280cc92-f432-45bd-ad3b-29d594831748 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:46:00.209743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280cc92-f432-45bd-ad3b-29d594831748 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:46:00.271474 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:46:00.307519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf917e8d-dff3-4f34-ae08-26ed5a0b165b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:46:00.308194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf917e8d-dff3-4f34-ae08-26ed5a0b165b admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:00.308416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf917e8d-dff3-4f34-ae08-26ed5a0b165b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:00.308687 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf917e8d-dff3-4f34-ae08-26ed5a0b165b admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:00.309601 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf917e8d-dff3-4f34-ae08-26ed5a0b165b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:46:00.311388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: WARNING py.warnings [None req-cf917e8d-dff3-4f34-ae08-26ed5a0b165b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:46:00.311388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: warnings.warn(msg) Aug 05 19:46:00.311388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: Aug 05 19:46:00.312585 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf917e8d-dff3-4f34-ae08-26ed5a0b165b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:46:00.372484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 737/1497] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:46:00 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:46:00.376648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc81bb3-c9fe-4ba2-9ba5-4aa9d22110fc admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:46:00.378156 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-dcc81bb3-c9fe-4ba2-9ba5-4aa9d22110fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:00.412365 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc81bb3-c9fe-4ba2-9ba5-4aa9d22110fc admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:46:00.509023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f32f66-1e7e-43bc-b562-cde259a19fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:46:00.509636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f32f66-1e7e-43bc-b562-cde259a19fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:00.509907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f32f66-1e7e-43bc-b562-cde259a19fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:00.510174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f32f66-1e7e-43bc-b562-cde259a19fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:00.510869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f32f66-1e7e-43bc-b562-cde259a19fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:46:00.511866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: WARNING py.warnings [None req-e4f32f66-1e7e-43bc-b562-cde259a19fd6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:46:00.511866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: warnings.warn(msg) Aug 05 19:46:00.511866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: Aug 05 19:46:00.513116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f32f66-1e7e-43bc-b562-cde259a19fd6 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:46:00.555884 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 761/1498] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:46:00 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:46:00.560342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf917e8d-dff3-4f34-ae08-26ed5a0b165b admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:46:00.561861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-cf917e8d-dff3-4f34-ae08-26ed5a0b165b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:00.624208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf917e8d-dff3-4f34-ae08-26ed5a0b165b admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:46:00.711608 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:46:00.723734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133536e-5cfd-438d-9803-24a90ebb6fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:46:00.724344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133536e-5cfd-438d-9803-24a90ebb6fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:00.724616 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133536e-5cfd-438d-9803-24a90ebb6fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:00.724882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133536e-5cfd-438d-9803-24a90ebb6fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:00.725568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133536e-5cfd-438d-9803-24a90ebb6fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:46:00.726605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: WARNING py.warnings [None req-b133536e-5cfd-438d-9803-24a90ebb6fe0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:46:00.726605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: warnings.warn(msg) Aug 05 19:46:00.726605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: Aug 05 19:46:00.727503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133536e-5cfd-438d-9803-24a90ebb6fe0 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:46:00.741242 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 738/1499] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:46:00.745385 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f32f66-1e7e-43bc-b562-cde259a19fd6 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:46:00.746970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e4f32f66-1e7e-43bc-b562-cde259a19fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:00.759339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f32f66-1e7e-43bc-b562-cde259a19fd6 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:46:00.784031 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:46:00.855147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0d0a7-2caa-46e2-b590-45fcff76e83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:46:00.855673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0d0a7-2caa-46e2-b590-45fcff76e83e admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:00.855947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0d0a7-2caa-46e2-b590-45fcff76e83e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:00.856207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0d0a7-2caa-46e2-b590-45fcff76e83e admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:00.856888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0d0a7-2caa-46e2-b590-45fcff76e83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:46:00.857889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: WARNING py.warnings [None req-1db0d0a7-2caa-46e2-b590-45fcff76e83e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:46:00.857889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: warnings.warn(msg) Aug 05 19:46:00.857889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: Aug 05 19:46:00.858809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0d0a7-2caa-46e2-b590-45fcff76e83e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:46:00.872080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 762/1500] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:46:00.876476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133536e-5cfd-438d-9803-24a90ebb6fe0 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:46:00.877982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b133536e-5cfd-438d-9803-24a90ebb6fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:00.913347 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133536e-5cfd-438d-9803-24a90ebb6fe0 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:46:01.007338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50823a-e9f2-46cf-9323-e0b83fdb606d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:46:01.008001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50823a-e9f2-46cf-9323-e0b83fdb606d admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:01.008295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50823a-e9f2-46cf-9323-e0b83fdb606d admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:01.008565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50823a-e9f2-46cf-9323-e0b83fdb606d admin admin] PATH_INFO: `/v3/projects/34ceab2be1b241ff96627302a3e640e7/users/ac88e00803954189a5c2591a341c30b6/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:01.031382 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50823a-e9f2-46cf-9323-e0b83fdb606d admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ceab2be1b241ff96627302a3e640e7, user_id=ac88e00803954189a5c2591a341c30b6, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:46:01.032418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: WARNING py.warnings [None req-9f50823a-e9f2-46cf-9323-e0b83fdb606d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:46:01.032418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: warnings.warn(msg) Aug 05 19:46:01.032418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: Aug 05 19:46:01.034351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50823a-e9f2-46cf-9323-e0b83fdb606d admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:46:01.059735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 739/1501] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:46:00 2022] PUT /identity/v3/projects/34ceab2be1b241ff96627302a3e640e7/users/ac88e00803954189a5c2591a341c30b6/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:46:01.072742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503c7b6-795c-4fc2-9348-23eb7c9008a6 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:01.073031 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503c7b6-795c-4fc2-9348-23eb7c9008a6 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:01.073309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503c7b6-795c-4fc2-9348-23eb7c9008a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:01.132440 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-d503c7b6-795c-4fc2-9348-23eb7c9008a6 None None] MFA Rules not processed for user `ac88e00803954189a5c2591a341c30b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:46:01.133610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d503c7b6-795c-4fc2-9348-23eb7c9008a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:01.213311 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 763/1502] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:46:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:46:01.227278 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603da53c-8219-4c9e-9f1d-938faec9bfe5 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:01.227501 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603da53c-8219-4c9e-9f1d-938faec9bfe5 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:01.227666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603da53c-8219-4c9e-9f1d-938faec9bfe5 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:01.229012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 740/1503] 10.222.0.86 () {54 vars in 964 bytes} [Fri Aug 5 19:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:46:01.233867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8badaf-f044-4346-a506-ac538b39f82e None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:01.234147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8badaf-f044-4346-a506-ac538b39f82e None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:01.234420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8badaf-f044-4346-a506-ac538b39f82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:01.266991 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:46:01.313972 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-0f8badaf-f044-4346-a506-ac538b39f82e None None] MFA Rules not processed for user `afea7d279292424490beff0481ff1512`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:46:01.315222 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-0f8badaf-f044-4346-a506-ac538b39f82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:01.393554 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 764/1504] 10.222.0.86 () {58 vars in 1061 bytes} [Fri Aug 5 19:46:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:46:01.401380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603da53c-8219-4c9e-9f1d-938faec9bfe5 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:46:01.402926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-603da53c-8219-4c9e-9f1d-938faec9bfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:01.454267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:46:01.513357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603da53c-8219-4c9e-9f1d-938faec9bfe5 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:46:01.586970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b102b0b-5c07-40aa-a1b1-b7f45033b0e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:46:01.587547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b102b0b-5c07-40aa-a1b1-b7f45033b0e1 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:01.587820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b102b0b-5c07-40aa-a1b1-b7f45033b0e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:01.588085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b102b0b-5c07-40aa-a1b1-b7f45033b0e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:01.589716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6b102b0b-5c07-40aa-a1b1-b7f45033b0e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:01.688144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b102b0b-5c07-40aa-a1b1-b7f45033b0e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:46:01.689489 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b102b0b-5c07-40aa-a1b1-b7f45033b0e1 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:46:01.693374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:46:01.708857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 741/1505] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:46:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:46:01.753231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c091a-800c-4918-b99f-018b14c72640 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:01.754286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c091a-800c-4918-b99f-018b14c72640 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:01.754286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c091a-800c-4918-b99f-018b14c72640 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:01.755422 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 765/1506] 10.222.0.86 () {52 vars in 830 bytes} [Fri Aug 5 19:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:46:01.759291 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b102b0b-5c07-40aa-a1b1-b7f45033b0e1 service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:46:01.760519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6b102b0b-5c07-40aa-a1b1-b7f45033b0e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:01.861530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b102b0b-5c07-40aa-a1b1-b7f45033b0e1 service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:46:01.955494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:46:01.956195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:01.956319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:01.956589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] PATH_INFO: `/v3/users/ac88e00803954189a5c2591a341c30b6` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:01.969902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac88e00803954189a5c2591a341c30b6)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:46:01.970943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: WARNING py.warnings [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:46:01.970943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: warnings.warn(msg) Aug 05 19:46:01.970943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: Aug 05 19:46:01.972269 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:46:01.981856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:46:02.039308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac88e00803954189a5c2591a341c30b6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:46:02.053130 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac88e00803954189a5c2591a341c30b6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:46:02.053689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac88e00803954189a5c2591a341c30b6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:46:02.076627 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac88e00803954189a5c2591a341c30b6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:46:02.086860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac88e00803954189a5c2591a341c30b6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:46:02.092900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 742/1507] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:46:01 2022] DELETE /identity/v3/users/ac88e00803954189a5c2591a341c30b6 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:46:02.097951 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e594ed3-1f03-4b71-94ff-633850afc73b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:02.098280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e594ed3-1f03-4b71-94ff-633850afc73b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:02.098555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e594ed3-1f03-4b71-94ff-633850afc73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:02.155933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-7e594ed3-1f03-4b71-94ff-633850afc73b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:46:02.157138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7e594ed3-1f03-4b71-94ff-633850afc73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:02.235119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:46:02.249181 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 766/1508] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:46:02.259016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:46:02.260232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:02.362031 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994cb25-d882-4d25-a503-9ca85223ef49 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:46:02.420819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:46:02.442662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adb964-167a-423f-8508-f2cc5b3c51ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:46:02.443371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adb964-167a-423f-8508-f2cc5b3c51ab service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:02.443736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adb964-167a-423f-8508-f2cc5b3c51ab service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:02.443996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adb964-167a-423f-8508-f2cc5b3c51ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:02.445624 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a0adb964-167a-423f-8508-f2cc5b3c51ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:02.556111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0adb964-167a-423f-8508-f2cc5b3c51ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:46:02.557534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0adb964-167a-423f-8508-f2cc5b3c51ab service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:46:02.577073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 743/1509] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:46:02.916416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e594ed3-1f03-4b71-94ff-633850afc73b None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:46:02.917860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7e594ed3-1f03-4b71-94ff-633850afc73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:46:02.930577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e594ed3-1f03-4b71-94ff-633850afc73b None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:46:03.029174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:46:03.029814 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:46:03.030090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:46:03.030359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4 admin admin] PATH_INFO: `/v3/projects/34ceab2be1b241ff96627302a3e640e7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:46:03.040675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34ceab2be1b241ff96627302a3e640e7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:46:03.041698 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: WARNING py.warnings [None req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:46:03.041698 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: warnings.warn(msg) Aug 05 19:46:03.041698 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: Aug 05 19:46:03.042927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:46:03.101769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34ceab2be1b241ff96627302a3e640e7', 'request_id': 'req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:46:03.115166 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34ceab2be1b241ff96627302a3e640e7', 'request_id': 'req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:46:03.124926 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34ceab2be1b241ff96627302a3e640e7', 'request_id': 'req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:46:03.148706 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4 admin admin] The token cache is being invalidate because project 34ceab2be1b241ff96627302a3e640e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:46:03.149884 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:46:03.149884 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fe9666eb-a27a-4e0a-a8de-e4ea689bcfa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:46:03.150867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 767/1510] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:46:02 2022] DELETE /identity/v3/projects/34ceab2be1b241ff96627302a3e640e7 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:46:12.574695 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=62 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:46:13.146694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:47:37.587399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1148490-1426-4163-a013-a79f3a2939ac None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:37.588018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1148490-1426-4163-a013-a79f3a2939ac None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:37.588018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1148490-1426-4163-a013-a79f3a2939ac None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:37.589208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:47:37.594333 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedbf3d-eb72-4e5b-ab0c-dba15adf87c9 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:37.595887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedbf3d-eb72-4e5b-ab0c-dba15adf87c9 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:37.595887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedbf3d-eb72-4e5b-ab0c-dba15adf87c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:37.684975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-0cedbf3d-eb72-4e5b-ab0c-dba15adf87c9 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:47:37.686195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-0cedbf3d-eb72-4e5b-ab0c-dba15adf87c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:47:37.795473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:47:37.963923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1148490-1426-4163-a013-a79f3a2939ac None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:47:37.966313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e1148490-1426-4163-a013-a79f3a2939ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:47:38.103535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1148490-1426-4163-a013-a79f3a2939ac None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:47:38.109288 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7584fde-4972-4dd7-a554-3c5d13693595 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:47:38.109961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7584fde-4972-4dd7-a554-3c5d13693595 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:38.110315 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7584fde-4972-4dd7-a554-3c5d13693595 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:38.110590 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7584fde-4972-4dd7-a554-3c5d13693595 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:38.120365 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7584fde-4972-4dd7-a554-3c5d13693595 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:47:38.121909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7584fde-4972-4dd7-a554-3c5d13693595 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:47:38.134157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 745/1513] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:47:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:47:39.008540 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f8de9-f32f-403e-882b-a7cd19237a99 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:39.009032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f8de9-f32f-403e-882b-a7cd19237a99 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:39.009146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f8de9-f32f-403e-882b-a7cd19237a99 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:39.010447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 769/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:47:39.015501 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e998f7-8829-42d9-a79a-90a5e660377b None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:39.015958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e998f7-8829-42d9-a79a-90a5e660377b None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:39.016579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e998f7-8829-42d9-a79a-90a5e660377b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:39.037152 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-40e998f7-8829-42d9-a79a-90a5e660377b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:47:39.038279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-40e998f7-8829-42d9-a79a-90a5e660377b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:47:39.046150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 746/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:47:39.201465 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f8de9-f32f-403e-882b-a7cd19237a99 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:47:39.216458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f8de9-f32f-403e-882b-a7cd19237a99 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:47:39.222906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f73bc0-34c0-462f-8307-4f41a7194a51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:47:39.223509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f73bc0-34c0-462f-8307-4f41a7194a51 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:39.223785 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f73bc0-34c0-462f-8307-4f41a7194a51 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:39.224056 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f73bc0-34c0-462f-8307-4f41a7194a51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:39.233049 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f73bc0-34c0-462f-8307-4f41a7194a51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:47:39.234942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f73bc0-34c0-462f-8307-4f41a7194a51 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:47:39.245158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 770/1516] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:47:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:47:39.343765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9dbcd-888a-4718-b082-6a1e41db5c1c None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:39.344065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9dbcd-888a-4718-b082-6a1e41db5c1c None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:39.344298 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9dbcd-888a-4718-b082-6a1e41db5c1c None None] PATH_INFO: `/v3` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:39.346407 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 747/1517] 10.222.0.79 () {54 vars in 925 bytes} [Fri Aug 5 19:47:39 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:47:39.352838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf9643-2d5f-4901-ade4-52c7ac615d2b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:39.353122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf9643-2d5f-4901-ade4-52c7ac615d2b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:39.353390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf9643-2d5f-4901-ade4-52c7ac615d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:39.422386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c7cf9643-2d5f-4901-ade4-52c7ac615d2b None None] MFA Rules not processed for user `cce71f93c80d433b830c448d0a7a14e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:47:39.423704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c7cf9643-2d5f-4901-ade4-52c7ac615d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:47:39.441443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:47:39.496083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 771/1518] 10.222.0.79 () {58 vars in 1015 bytes} [Fri Aug 5 19:47:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:47:43.308321 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc89155-4a59-4dbe-8a38-45ec5cd94aa2 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:43.309563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc89155-4a59-4dbe-8a38-45ec5cd94aa2 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:43.310025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc89155-4a59-4dbe-8a38-45ec5cd94aa2 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:43.312157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 748/1519] 10.222.0.86 () {54 vars in 961 bytes} [Fri Aug 5 19:47:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:47:43.318402 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6f9f4-305e-41aa-a4a1-73f084cd44c5 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:43.318913 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6f9f4-305e-41aa-a4a1-73f084cd44c5 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:43.319026 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6f9f4-305e-41aa-a4a1-73f084cd44c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:43.339956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-32b6f9f4-305e-41aa-a4a1-73f084cd44c5 None None] MFA Rules not processed for user `cce71f93c80d433b830c448d0a7a14e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:47:43.341217 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-32b6f9f4-305e-41aa-a4a1-73f084cd44c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:47:43.350119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 772/1520] 10.222.0.86 () {58 vars in 1058 bytes} [Fri Aug 5 19:47:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:47:43.357668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc89155-4a59-4dbe-8a38-45ec5cd94aa2 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:47:43.371611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc89155-4a59-4dbe-8a38-45ec5cd94aa2 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:47:43.377180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afcf4d-b1e5-465a-9e71-4cb49a0f0294 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:47:43.377782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afcf4d-b1e5-465a-9e71-4cb49a0f0294 service placement] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:43.378178 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afcf4d-b1e5-465a-9e71-4cb49a0f0294 service placement] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:43.378454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afcf4d-b1e5-465a-9e71-4cb49a0f0294 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:43.387063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afcf4d-b1e5-465a-9e71-4cb49a0f0294 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:47:43.388646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afcf4d-b1e5-465a-9e71-4cb49a0f0294 service placement] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:47:43.398705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 749/1521] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:47:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:47:50.496214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98617d7d-89bf-44d4-b2d7-33c8485f2a1d None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:50.496765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98617d7d-89bf-44d4-b2d7-33c8485f2a1d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:50.496765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98617d7d-89bf-44d4-b2d7-33c8485f2a1d None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:50.497922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:47:50.503009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a15040-d9ef-4d0f-9f3c-a2e5b5f9ce30 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:50.503499 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a15040-d9ef-4d0f-9f3c-a2e5b5f9ce30 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:50.503614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a15040-d9ef-4d0f-9f3c-a2e5b5f9ce30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:50.523366 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c5a15040-d9ef-4d0f-9f3c-a2e5b5f9ce30 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:47:50.524567 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c5a15040-d9ef-4d0f-9f3c-a2e5b5f9ce30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:47:50.533324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:47:50.689090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98617d7d-89bf-44d4-b2d7-33c8485f2a1d None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:47:50.703574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98617d7d-89bf-44d4-b2d7-33c8485f2a1d None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:47:50.709417 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c79be3-7650-4bb5-ab65-f064167fb696 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:47:50.710047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c79be3-7650-4bb5-ab65-f064167fb696 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:50.710320 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c79be3-7650-4bb5-ab65-f064167fb696 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:50.710585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c79be3-7650-4bb5-ab65-f064167fb696 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:50.718766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c79be3-7650-4bb5-ab65-f064167fb696 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:47:50.721014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c79be3-7650-4bb5-ab65-f064167fb696 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:47:50.730771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 774/1524] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:47:51.554437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb66f6-23e0-4609-9ce9-c85654f692dd None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:51.555045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb66f6-23e0-4609-9ce9-c85654f692dd None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:51.555045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb66f6-23e0-4609-9ce9-c85654f692dd None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:51.556371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 5 19:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:47:51.561601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd690733-fcf4-40ea-80e2-f95a5f0f24ed None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:51.561992 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd690733-fcf4-40ea-80e2-f95a5f0f24ed None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:51.562167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd690733-fcf4-40ea-80e2-f95a5f0f24ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:51.582114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-cd690733-fcf4-40ea-80e2-f95a5f0f24ed None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:47:51.583325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-cd690733-fcf4-40ea-80e2-f95a5f0f24ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:47:51.591500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 5 19:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:47:51.747265 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb66f6-23e0-4609-9ce9-c85654f692dd None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:47:51.761556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb66f6-23e0-4609-9ce9-c85654f692dd None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:47:51.767649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57626029-5f40-4074-8628-9733b3beaed3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:47:51.768251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57626029-5f40-4074-8628-9733b3beaed3 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:51.768538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57626029-5f40-4074-8628-9733b3beaed3 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:51.768954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57626029-5f40-4074-8628-9733b3beaed3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:51.779761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57626029-5f40-4074-8628-9733b3beaed3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:47:51.782092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57626029-5f40-4074-8628-9733b3beaed3 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:47:51.791712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 752/1527] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:47:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:47:55.828426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b4606-3568-4491-ae93-130f184873da None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:55.828978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b4606-3568-4491-ae93-130f184873da None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:55.828978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b4606-3568-4491-ae93-130f184873da None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:55.830325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:47:55.835629 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60251806-a544-450d-88ce-72f60ecc9ceb None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:55.836091 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60251806-a544-450d-88ce-72f60ecc9ceb None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:55.836199 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60251806-a544-450d-88ce-72f60ecc9ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:55.855896 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-60251806-a544-450d-88ce-72f60ecc9ceb None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:47:55.857083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-60251806-a544-450d-88ce-72f60ecc9ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:47:55.865659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:47:55.941903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b4606-3568-4491-ae93-130f184873da None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:47:55.943780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f07b4606-3568-4491-ae93-130f184873da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:47:56.027179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b4606-3568-4491-ae93-130f184873da None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:47:56.030769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a34b36-9e8f-4262-8840-159f75042bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:47:56.031909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a34b36-9e8f-4262-8840-159f75042bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:56.032542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a34b36-9e8f-4262-8840-159f75042bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:56.032657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a34b36-9e8f-4262-8840-159f75042bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:56.042252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a34b36-9e8f-4262-8840-159f75042bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:47:56.043726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a34b36-9e8f-4262-8840-159f75042bb8 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:47:56.054798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 777/1530] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:47:58.170147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10037bb3-9290-4aa7-9368-646abaf2597e None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:58.170717 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10037bb3-9290-4aa7-9368-646abaf2597e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:58.170829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10037bb3-9290-4aa7-9368-646abaf2597e None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:58.172879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:47:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:47:58.178746 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05fd5a-6c76-4f54-99c5-1aadd2ea62ae None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:58.179243 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05fd5a-6c76-4f54-99c5-1aadd2ea62ae None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:58.179339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05fd5a-6c76-4f54-99c5-1aadd2ea62ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:58.199866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3e05fd5a-6c76-4f54-99c5-1aadd2ea62ae None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:47:58.201063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3e05fd5a-6c76-4f54-99c5-1aadd2ea62ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:47:58.209302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:47:58.246646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10037bb3-9290-4aa7-9368-646abaf2597e None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:47:58.258507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10037bb3-9290-4aa7-9368-646abaf2597e None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:47:58.262133 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e21f2-3072-4c1e-aa89-81dd91eaeb14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:47:58.262776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e21f2-3072-4c1e-aa89-81dd91eaeb14 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:47:58.263049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e21f2-3072-4c1e-aa89-81dd91eaeb14 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:47:58.263315 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e21f2-3072-4c1e-aa89-81dd91eaeb14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:47:58.273082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e21f2-3072-4c1e-aa89-81dd91eaeb14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:47:58.274501 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e21f2-3072-4c1e-aa89-81dd91eaeb14 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:47:58.285944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 755/1533] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:47:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:48:01.993958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f85691-2d93-48d4-a0ae-c382c3f55818 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:01.994553 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f85691-2d93-48d4-a0ae-c382c3f55818 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:01.994553 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f85691-2d93-48d4-a0ae-c382c3f55818 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:01.995899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:02.001876 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc01859-dc29-4b77-8c50-c4b65b31bdb6 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:02.002347 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc01859-dc29-4b77-8c50-c4b65b31bdb6 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:02.002468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc01859-dc29-4b77-8c50-c4b65b31bdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:02.023868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-7dc01859-dc29-4b77-8c50-c4b65b31bdb6 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:02.025093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7dc01859-dc29-4b77-8c50-c4b65b31bdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:02.033520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:48:02.350810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f85691-2d93-48d4-a0ae-c382c3f55818 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:02.352522 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b2f85691-2d93-48d4-a0ae-c382c3f55818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:02.417925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:02.432137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f85691-2d93-48d4-a0ae-c382c3f55818 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:02.435948 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74686a0b-644c-4e6e-94ee-0d32b829ce06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:02.436557 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686a0b-644c-4e6e-94ee-0d32b829ce06 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:02.436828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686a0b-644c-4e6e-94ee-0d32b829ce06 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:02.437860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686a0b-644c-4e6e-94ee-0d32b829ce06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:02.447189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74686a0b-644c-4e6e-94ee-0d32b829ce06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:02.448643 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74686a0b-644c-4e6e-94ee-0d32b829ce06 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:02.458160 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 780/1536] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:48:04.542330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7c5e6-edd6-40cd-983a-89b7efce98e1 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:04.542914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7c5e6-edd6-40cd-983a-89b7efce98e1 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:04.542914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7c5e6-edd6-40cd-983a-89b7efce98e1 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:04.544262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 5 19:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:04.549836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f505d0-307f-4755-acc5-baf9058adad8 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:04.550276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f505d0-307f-4755-acc5-baf9058adad8 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:04.550381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f505d0-307f-4755-acc5-baf9058adad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:04.569856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-08f505d0-307f-4755-acc5-baf9058adad8 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:04.571080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-08f505d0-307f-4755-acc5-baf9058adad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:04.579321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 5 19:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:48:04.886499 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7c5e6-edd6-40cd-983a-89b7efce98e1 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:04.888197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9aa7c5e6-edd6-40cd-983a-89b7efce98e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:04.896705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=84 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:04.903129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7c5e6-edd6-40cd-983a-89b7efce98e1 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:04.907103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc1309-16a3-40d8-b408-593920ea48d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:04.907708 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc1309-16a3-40d8-b408-593920ea48d0 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:04.907987 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc1309-16a3-40d8-b408-593920ea48d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:04.908259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc1309-16a3-40d8-b408-593920ea48d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:04.917194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc1309-16a3-40d8-b408-593920ea48d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:04.918607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc1309-16a3-40d8-b408-593920ea48d0 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:04.928175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 758/1539] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:48:14.564778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=11 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:14.924064 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=5 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:32.228503 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4634-86e2-492a-9834-4e6f7aa662c0 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.229407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4634-86e2-492a-9834-4e6f7aa662c0 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.229407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4634-86e2-492a-9834-4e6f7aa662c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.256739 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-9d2c4634-86e2-492a-9834-4e6f7aa662c0 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.258126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9d2c4634-86e2-492a-9834-4e6f7aa662c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.271351 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 782/1540] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:32.272084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0645d-4429-45db-bc17-21fbab1c1e41 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.276098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0645d-4429-45db-bc17-21fbab1c1e41 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.276098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0645d-4429-45db-bc17-21fbab1c1e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.282684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cc742-bddd-4311-afba-04362c4fa292 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.282684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cc742-bddd-4311-afba-04362c4fa292 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.282684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cc742-bddd-4311-afba-04362c4fa292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.308171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-e3e0645d-4429-45db-bc17-21fbab1c1e41 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.310646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e3e0645d-4429-45db-bc17-21fbab1c1e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.314502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c98cc742-bddd-4311-afba-04362c4fa292 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.315770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c98cc742-bddd-4311-afba-04362c4fa292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.328470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 759/1541] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:32.337998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 783/1542] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:49:32.343633 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0645d-4429-45db-bc17-21fbab1c1e41 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.347259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fcc0b-2ed4-40eb-8122-f417ae66c903 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.347716 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fcc0b-2ed4-40eb-8122-f417ae66c903 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.347851 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fcc0b-2ed4-40eb-8122-f417ae66c903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.363198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0645d-4429-45db-bc17-21fbab1c1e41 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.374458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58a4aa-cdb2-4789-ab39-f94564d05cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.376197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58a4aa-cdb2-4789-ab39-f94564d05cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.376731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58a4aa-cdb2-4789-ab39-f94564d05cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.377245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58a4aa-cdb2-4789-ab39-f94564d05cf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.378233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58a4aa-cdb2-4789-ab39-f94564d05cf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:32.378814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58a4aa-cdb2-4789-ab39-f94564d05cf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.380094 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: WARNING py.warnings [None req-dc58a4aa-cdb2-4789-ab39-f94564d05cf6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:49:32.380094 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: warnings.warn(msg) Aug 05 19:49:32.380094 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: Aug 05 19:49:32.381483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58a4aa-cdb2-4789-ab39-f94564d05cf6 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.392195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-d45fcc0b-2ed4-40eb-8122-f417ae66c903 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.393742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d45fcc0b-2ed4-40eb-8122-f417ae66c903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.401475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 760/1543] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:32.407476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bd011-6880-40c1-9792-56b58dd76ae2 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.408168 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bd011-6880-40c1-9792-56b58dd76ae2 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.408679 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bd011-6880-40c1-9792-56b58dd76ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.413753 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 784/1544] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:49:32.421348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fcc0b-2ed4-40eb-8122-f417ae66c903 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.438665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fcc0b-2ed4-40eb-8122-f417ae66c903 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.447773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904fe77-a879-4182-8ccd-5854983ee471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.448228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-793bd011-6880-40c1-9792-56b58dd76ae2 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.448426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904fe77-a879-4182-8ccd-5854983ee471 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.448707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904fe77-a879-4182-8ccd-5854983ee471 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.448975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904fe77-a879-4182-8ccd-5854983ee471 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.450212 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904fe77-a879-4182-8ccd-5854983ee471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:32.450668 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904fe77-a879-4182-8ccd-5854983ee471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.451732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-793bd011-6880-40c1-9792-56b58dd76ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.453108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904fe77-a879-4182-8ccd-5854983ee471 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.475073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 761/1545] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:32.476305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bd011-6880-40c1-9792-56b58dd76ae2 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.484943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 785/1546] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:32.488155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1ba08-0c63-4bfd-809c-f55e26082708 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.488444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1ba08-0c63-4bfd-809c-f55e26082708 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.488725 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1ba08-0c63-4bfd-809c-f55e26082708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.493070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bd011-6880-40c1-9792-56b58dd76ae2 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.502824 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5725da-32e4-4ab4-b386-e97180ab6268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.503730 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5725da-32e4-4ab4-b386-e97180ab6268 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.504155 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5725da-32e4-4ab4-b386-e97180ab6268 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.504609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5725da-32e4-4ab4-b386-e97180ab6268 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.505453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5725da-32e4-4ab4-b386-e97180ab6268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.507785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5725da-32e4-4ab4-b386-e97180ab6268 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.536268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-dee1ba08-0c63-4bfd-809c-f55e26082708 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.537639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-dee1ba08-0c63-4bfd-809c-f55e26082708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.559001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 786/1547] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:32.569967 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f91d3-e4ef-43d4-83a0-80c25a632a90 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.571271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f91d3-e4ef-43d4-83a0-80c25a632a90 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.572980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f91d3-e4ef-43d4-83a0-80c25a632a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.577124 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 762/1548] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:32.578248 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5725da-32e4-4ab4-b386-e97180ab6268 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.619904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-0c4f91d3-e4ef-43d4-83a0-80c25a632a90 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.621525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-0c4f91d3-e4ef-43d4-83a0-80c25a632a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.691284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5725da-32e4-4ab4-b386-e97180ab6268 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.696370 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73fd27-026e-4e75-bc4a-2c913f9add6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.696998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73fd27-026e-4e75-bc4a-2c913f9add6b admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.697276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73fd27-026e-4e75-bc4a-2c913f9add6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.697755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73fd27-026e-4e75-bc4a-2c913f9add6b admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.698423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73fd27-026e-4e75-bc4a-2c913f9add6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.700899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73fd27-026e-4e75-bc4a-2c913f9add6b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.713232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 787/1549] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:32.714098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f91d3-e4ef-43d4-83a0-80c25a632a90 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.727757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f91d3-e4ef-43d4-83a0-80c25a632a90 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.735361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5993b0-4cf2-4e71-9bc4-02f3ed11a847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.736491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5993b0-4cf2-4e71-9bc4-02f3ed11a847 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.737014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5993b0-4cf2-4e71-9bc4-02f3ed11a847 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.737581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5993b0-4cf2-4e71-9bc4-02f3ed11a847 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.738292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5993b0-4cf2-4e71-9bc4-02f3ed11a847 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:32.738764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5993b0-4cf2-4e71-9bc4-02f3ed11a847 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.747249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5993b0-4cf2-4e71-9bc4-02f3ed11a847 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.750458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 763/1550] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:32.752754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b44f4e-f2d1-4c83-8b70-832a39741117 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.753053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b44f4e-f2d1-4c83-8b70-832a39741117 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.753332 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b44f4e-f2d1-4c83-8b70-832a39741117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.754947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 788/1551] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:32.755740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5993b0-4cf2-4e71-9bc4-02f3ed11a847 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.766118 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5993b0-4cf2-4e71-9bc4-02f3ed11a847 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.775598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-45b44f4e-f2d1-4c83-8b70-832a39741117 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.776830 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-45b44f4e-f2d1-4c83-8b70-832a39741117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.844021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b40234-0c26-4278-9d10-bb5f6b20ad1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.844872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b40234-0c26-4278-9d10-bb5f6b20ad1a admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.845255 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b40234-0c26-4278-9d10-bb5f6b20ad1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.845724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b40234-0c26-4278-9d10-bb5f6b20ad1a admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.846677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b40234-0c26-4278-9d10-bb5f6b20ad1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.849035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b40234-0c26-4278-9d10-bb5f6b20ad1a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.854464 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 764/1552] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:32.855597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308718ee-dbbd-4442-8036-bc513544201a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.856076 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308718ee-dbbd-4442-8036-bc513544201a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.857311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308718ee-dbbd-4442-8036-bc513544201a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.882694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 789/1553] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:32.882694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab4fa2-0ef0-4ec3-8af3-ccd8313bcbcc None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.882694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab4fa2-0ef0-4ec3-8af3-ccd8313bcbcc None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.882694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab4fa2-0ef0-4ec3-8af3-ccd8313bcbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.885867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-308718ee-dbbd-4442-8036-bc513544201a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.887543 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-308718ee-dbbd-4442-8036-bc513544201a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.897286 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 765/1554] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:32.898973 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717942d9-e7d1-4e59-9362-649805f1bfca None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.899119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717942d9-e7d1-4e59-9362-649805f1bfca None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.899440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717942d9-e7d1-4e59-9362-649805f1bfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.905152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-27ab4fa2-0ef0-4ec3-8af3-ccd8313bcbcc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.906397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-27ab4fa2-0ef0-4ec3-8af3-ccd8313bcbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.915496 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 790/1555] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:32.916938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cd066-8909-4949-8b9c-ee07e76528c3 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.917232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cd066-8909-4949-8b9c-ee07e76528c3 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.917498 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cd066-8909-4949-8b9c-ee07e76528c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.925877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-717942d9-e7d1-4e59-9362-649805f1bfca None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.927297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-717942d9-e7d1-4e59-9362-649805f1bfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.940470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 766/1556] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:32.941275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717942d9-e7d1-4e59-9362-649805f1bfca None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.945042 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-693cd066-8909-4949-8b9c-ee07e76528c3 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.946261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-693cd066-8909-4949-8b9c-ee07e76528c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.952780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717942d9-e7d1-4e59-9362-649805f1bfca None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.955914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 791/1557] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:32.960003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cd066-8909-4949-8b9c-ee07e76528c3 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.960408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9021b-d4f1-4a5f-8a92-cfdb32570c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.961403 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9021b-d4f1-4a5f-8a92-cfdb32570c91 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.961929 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9021b-d4f1-4a5f-8a92-cfdb32570c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.962338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9021b-d4f1-4a5f-8a92-cfdb32570c91 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.963650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9021b-d4f1-4a5f-8a92-cfdb32570c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.966146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: WARNING py.warnings [None req-0ba9021b-d4f1-4a5f-8a92-cfdb32570c91 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:49:32.966146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: warnings.warn(msg) Aug 05 19:49:32.966146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: Aug 05 19:49:32.968557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9021b-d4f1-4a5f-8a92-cfdb32570c91 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.976601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cd066-8909-4949-8b9c-ee07e76528c3 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.986141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7caa8-1450-47d2-bdda-4c2c1a55c89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.987169 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7caa8-1450-47d2-bdda-4c2c1a55c89a admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.988351 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7caa8-1450-47d2-bdda-4c2c1a55c89a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.988351 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7caa8-1450-47d2-bdda-4c2c1a55c89a admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.988964 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7caa8-1450-47d2-bdda-4c2c1a55c89a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.990409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: WARNING py.warnings [None req-58c7caa8-1450-47d2-bdda-4c2c1a55c89a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:49:32.990409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: warnings.warn(msg) Aug 05 19:49:32.990409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: Aug 05 19:49:32.992100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7caa8-1450-47d2-bdda-4c2c1a55c89a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.996901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:33.008718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 767/1558] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/users => generated 289 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:33.011745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d189ad8-2c75-43e8-8a6d-2f3897c5bad7 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.012013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d189ad8-2c75-43e8-8a6d-2f3897c5bad7 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.012458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d189ad8-2c75-43e8-8a6d-2f3897c5bad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.029781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 792/1559] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:33.030656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7caa8-1450-47d2-bdda-4c2c1a55c89a admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.035629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:33.036503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-8d189ad8-2c75-43e8-8a6d-2f3897c5bad7 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:33.037580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-8d189ad8-2c75-43e8-8a6d-2f3897c5bad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:33.103190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 768/1560] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:33.104832 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1f907-e8db-4e6e-ada3-c07c1489bf5e None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.105114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1f907-e8db-4e6e-ada3-c07c1489bf5e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.105383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1f907-e8db-4e6e-ada3-c07c1489bf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.109138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7caa8-1450-47d2-bdda-4c2c1a55c89a admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.114870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2863c-ee05-452d-88fa-ad3b567df9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.115527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2863c-ee05-452d-88fa-ad3b567df9af admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.115808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2863c-ee05-452d-88fa-ad3b567df9af admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.116240 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2863c-ee05-452d-88fa-ad3b567df9af admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.117259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2863c-ee05-452d-88fa-ad3b567df9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.118947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2863c-ee05-452d-88fa-ad3b567df9af admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.131984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-90a1f907-e8db-4e6e-ada3-c07c1489bf5e None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:33.132194 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 793/1561] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:33.132921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-90a1f907-e8db-4e6e-ada3-c07c1489bf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:33.133443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b3bdd-675c-423b-b68f-db6200f9b8d2 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.134557 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b3bdd-675c-423b-b68f-db6200f9b8d2 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.134557 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b3bdd-675c-423b-b68f-db6200f9b8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.142984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 769/1562] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:33.143820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1f907-e8db-4e6e-ada3-c07c1489bf5e None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.155207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1f907-e8db-4e6e-ada3-c07c1489bf5e None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.156402 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-5a7b3bdd-675c-423b-b68f-db6200f9b8d2 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:33.157780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5a7b3bdd-675c-423b-b68f-db6200f9b8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:33.159800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dba6b-9e02-46cf-9ded-d32a6c3f1802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.160408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dba6b-9e02-46cf-9ded-d32a6c3f1802 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.160681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dba6b-9e02-46cf-9ded-d32a6c3f1802 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.160947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dba6b-9e02-46cf-9ded-d32a6c3f1802 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.161634 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59dba6b-9e02-46cf-9ded-d32a6c3f1802 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:33.161930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59dba6b-9e02-46cf-9ded-d32a6c3f1802 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.163551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59dba6b-9e02-46cf-9ded-d32a6c3f1802 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.165821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 794/1563] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:33.167119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344d804-72dd-4c53-ba19-ecbf83616a1a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.167455 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344d804-72dd-4c53-ba19-ecbf83616a1a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.167704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344d804-72dd-4c53-ba19-ecbf83616a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.180730 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 770/1564] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:33.181577 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dba6b-9e02-46cf-9ded-d32a6c3f1802 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.188864 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-7344d804-72dd-4c53-ba19-ecbf83616a1a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:33.190181 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7344d804-72dd-4c53-ba19-ecbf83616a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:33.198220 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 795/1565] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:33.200313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dba6b-9e02-46cf-9ded-d32a6c3f1802 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.200662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344d804-72dd-4c53-ba19-ecbf83616a1a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.207556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254143b-13a2-452d-ae45-67d4919cb251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.208204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254143b-13a2-452d-ae45-67d4919cb251 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.208484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254143b-13a2-452d-ae45-67d4919cb251 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.208755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254143b-13a2-452d-ae45-67d4919cb251 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.209452 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254143b-13a2-452d-ae45-67d4919cb251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.210902 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344d804-72dd-4c53-ba19-ecbf83616a1a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.211087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254143b-13a2-452d-ae45-67d4919cb251 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.217175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb40ab5-b176-4031-bccc-408a4a4338e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.218234 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb40ab5-b176-4031-bccc-408a4a4338e4 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.218510 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb40ab5-b176-4031-bccc-408a4a4338e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.218811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb40ab5-b176-4031-bccc-408a4a4338e4 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.219640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb40ab5-b176-4031-bccc-408a4a4338e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.221559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb40ab5-b176-4031-bccc-408a4a4338e4 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.222861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 771/1566] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:33.223821 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254143b-13a2-452d-ae45-67d4919cb251 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.236156 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254143b-13a2-452d-ae45-67d4919cb251 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.242392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581d1f7-cd89-4276-b3bf-3e6a949dc836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.243065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581d1f7-cd89-4276-b3bf-3e6a949dc836 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.243330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581d1f7-cd89-4276-b3bf-3e6a949dc836 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.243598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581d1f7-cd89-4276-b3bf-3e6a949dc836 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.244318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581d1f7-cd89-4276-b3bf-3e6a949dc836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:33.244620 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581d1f7-cd89-4276-b3bf-3e6a949dc836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.246220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581d1f7-cd89-4276-b3bf-3e6a949dc836 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.247448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 796/1567] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:33.248857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb40ab5-b176-4031-bccc-408a4a4338e4 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.259434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb40ab5-b176-4031-bccc-408a4a4338e4 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.270157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 772/1568] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:33.270157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581d1f7-cd89-4276-b3bf-3e6a949dc836 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.270485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3266c5-a461-4b5b-a690-68e0fea30c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.270485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3266c5-a461-4b5b-a690-68e0fea30c52 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.270485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3266c5-a461-4b5b-a690-68e0fea30c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.270485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3266c5-a461-4b5b-a690-68e0fea30c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.270895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3266c5-a461-4b5b-a690-68e0fea30c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.270895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3266c5-a461-4b5b-a690-68e0fea30c52 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.277244 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581d1f7-cd89-4276-b3bf-3e6a949dc836 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.281943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 797/1569] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:33.282961 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3266c5-a461-4b5b-a690-68e0fea30c52 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.287703 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091135b4-de44-4d9e-9571-7630f6d3a67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.288327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091135b4-de44-4d9e-9571-7630f6d3a67f admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.288601 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091135b4-de44-4d9e-9571-7630f6d3a67f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.288867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091135b4-de44-4d9e-9571-7630f6d3a67f admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.289557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091135b4-de44-4d9e-9571-7630f6d3a67f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:33.289859 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091135b4-de44-4d9e-9571-7630f6d3a67f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.291527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091135b4-de44-4d9e-9571-7630f6d3a67f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.294227 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3266c5-a461-4b5b-a690-68e0fea30c52 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.299192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5182a7-a4d2-4260-9273-7c68929bbbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.299936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5182a7-a4d2-4260-9273-7c68929bbbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.300395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5182a7-a4d2-4260-9273-7c68929bbbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.300508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5182a7-a4d2-4260-9273-7c68929bbbc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.301274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5182a7-a4d2-4260-9273-7c68929bbbc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:33.301645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5182a7-a4d2-4260-9273-7c68929bbbc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.303381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5182a7-a4d2-4260-9273-7c68929bbbc1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.305014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 773/1570] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:33.305688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091135b4-de44-4d9e-9571-7630f6d3a67f admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.316399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091135b4-de44-4d9e-9571-7630f6d3a67f admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.317976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 798/1571] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:33.318903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5182a7-a4d2-4260-9273-7c68929bbbc1 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.329625 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5182a7-a4d2-4260-9273-7c68929bbbc1 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.331540 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dba0c-f9e9-4bcb-b6f2-382680b56e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.332168 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dba0c-f9e9-4bcb-b6f2-382680b56e36 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.332445 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dba0c-f9e9-4bcb-b6f2-382680b56e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.332715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dba0c-f9e9-4bcb-b6f2-382680b56e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.333385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dba0c-f9e9-4bcb-b6f2-382680b56e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.334382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34963ab6-ca65-4c3f-92c4-78e0a2a8092b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.335092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34963ab6-ca65-4c3f-92c4-78e0a2a8092b admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.335440 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34963ab6-ca65-4c3f-92c4-78e0a2a8092b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.335539 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dba0c-f9e9-4bcb-b6f2-382680b56e36 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.335832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34963ab6-ca65-4c3f-92c4-78e0a2a8092b admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.336588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34963ab6-ca65-4c3f-92c4-78e0a2a8092b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:33.336955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34963ab6-ca65-4c3f-92c4-78e0a2a8092b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.338594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34963ab6-ca65-4c3f-92c4-78e0a2a8092b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.352783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 799/1572] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:33.353929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34963ab6-ca65-4c3f-92c4-78e0a2a8092b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.365890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34963ab6-ca65-4c3f-92c4-78e0a2a8092b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.372413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 774/1573] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:33.372413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dba0c-f9e9-4bcb-b6f2-382680b56e36 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.438338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e4617-9dd9-40f2-bbcc-6735e1ebe08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.439030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e4617-9dd9-40f2-bbcc-6735e1ebe08d admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.439433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e4617-9dd9-40f2-bbcc-6735e1ebe08d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.439779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e4617-9dd9-40f2-bbcc-6735e1ebe08d admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.440490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e4617-9dd9-40f2-bbcc-6735e1ebe08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.442196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e4617-9dd9-40f2-bbcc-6735e1ebe08d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.444771 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dba0c-f9e9-4bcb-b6f2-382680b56e36 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.450045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cacc4-a55b-459a-bd8d-f0916d8e25f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.451047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cacc4-a55b-459a-bd8d-f0916d8e25f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.451319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cacc4-a55b-459a-bd8d-f0916d8e25f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.451588 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cacc4-a55b-459a-bd8d-f0916d8e25f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.452284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50cacc4-a55b-459a-bd8d-f0916d8e25f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.453925 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50cacc4-a55b-459a-bd8d-f0916d8e25f0 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.454151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 800/1574] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:33.455002 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e4617-9dd9-40f2-bbcc-6735e1ebe08d admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.468673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 775/1575] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:33.468673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cacc4-a55b-459a-bd8d-f0916d8e25f0 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.471416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e4617-9dd9-40f2-bbcc-6735e1ebe08d admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.476138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e9659-a4de-4499-a605-e22e9ae5d29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.476730 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cacc4-a55b-459a-bd8d-f0916d8e25f0 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.476915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e9659-a4de-4499-a605-e22e9ae5d29f admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.477054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e9659-a4de-4499-a605-e22e9ae5d29f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.477339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e9659-a4de-4499-a605-e22e9ae5d29f admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.477948 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e9659-a4de-4499-a605-e22e9ae5d29f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.479888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e9659-a4de-4499-a605-e22e9ae5d29f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.483164 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acabca-20f4-4467-9f39-5a7493788d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.483838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acabca-20f4-4467-9f39-5a7493788d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.484113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acabca-20f4-4467-9f39-5a7493788d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.484384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acabca-20f4-4467-9f39-5a7493788d45 admin admin] PATH_INFO: `/v3/projects/0b2146a5f18b4224b571bebf542704dd/users/46c447b35cd14422baaf480f22dc1ba3/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.498254 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:33.502546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acabca-20f4-4467-9f39-5a7493788d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2146a5f18b4224b571bebf542704dd, user_id=46c447b35cd14422baaf480f22dc1ba3, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.505976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acabca-20f4-4467-9f39-5a7493788d45 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.518665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 801/1576] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:33.518665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e9659-a4de-4499-a605-e22e9ae5d29f admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.522441 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:33.524180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 776/1577] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:33 2022] PUT /identity/v3/projects/0b2146a5f18b4224b571bebf542704dd/users/46c447b35cd14422baaf480f22dc1ba3/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:33.525142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acabca-20f4-4467-9f39-5a7493788d45 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.595540 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e9659-a4de-4499-a605-e22e9ae5d29f admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.599676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acabca-20f4-4467-9f39-5a7493788d45 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.600163 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ac522-f304-4ddf-b81f-18200017b9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.600786 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ac522-f304-4ddf-b81f-18200017b9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.601058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ac522-f304-4ddf-b81f-18200017b9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.601323 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ac522-f304-4ddf-b81f-18200017b9a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.601984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ac522-f304-4ddf-b81f-18200017b9a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.603993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ac522-f304-4ddf-b81f-18200017b9a4 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.604533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8d4d3-fe80-4303-ab76-588baa2bec63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.605146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8d4d3-fe80-4303-ab76-588baa2bec63 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.605424 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8d4d3-fe80-4303-ab76-588baa2bec63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.605721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8d4d3-fe80-4303-ab76-588baa2bec63 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.606411 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8d4d3-fe80-4303-ab76-588baa2bec63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.608355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8d4d3-fe80-4303-ab76-588baa2bec63 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.638981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 802/1578] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/projects => generated 389 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:33.643130 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ac522-f304-4ddf-b81f-18200017b9a4 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.647092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 777/1579] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:33.647092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8d4d3-fe80-4303-ab76-588baa2bec63 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.712404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8d4d3-fe80-4303-ab76-588baa2bec63 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.712876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ac522-f304-4ddf-b81f-18200017b9a4 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.717429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c7fa1-8dc4-4cb1-b505-28329811a65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.718157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c7fa1-8dc4-4cb1-b505-28329811a65f admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.718284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968af24-31bc-41ce-9bd1-6a21fee73798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.718436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c7fa1-8dc4-4cb1-b505-28329811a65f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.718794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c7fa1-8dc4-4cb1-b505-28329811a65f admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.718908 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968af24-31bc-41ce-9bd1-6a21fee73798 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.719069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968af24-31bc-41ce-9bd1-6a21fee73798 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.719340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968af24-31bc-41ce-9bd1-6a21fee73798 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.719444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c7fa1-8dc4-4cb1-b505-28329811a65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.720001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968af24-31bc-41ce-9bd1-6a21fee73798 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.721310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c7fa1-8dc4-4cb1-b505-28329811a65f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.721905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968af24-31bc-41ce-9bd1-6a21fee73798 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.751162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 778/1580] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:33.751162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c7fa1-8dc4-4cb1-b505-28329811a65f admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.753659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 803/1581] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/projects => generated 391 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:33.757541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968af24-31bc-41ce-9bd1-6a21fee73798 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.758734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c7fa1-8dc4-4cb1-b505-28329811a65f admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.831933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd751a-eb9a-4dd6-b61b-5df970e401b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.832626 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd751a-eb9a-4dd6-b61b-5df970e401b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.832901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd751a-eb9a-4dd6-b61b-5df970e401b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.833175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd751a-eb9a-4dd6-b61b-5df970e401b7 admin admin] PATH_INFO: `/v3/projects/d773fbf1acca4df9947f053573486931/users/a1d7a67f595b44a89abe80bc4f714821/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.835411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968af24-31bc-41ce-9bd1-6a21fee73798 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.840227 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ced9e-c1d0-45c4-bbd5-fc6d67652cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.841283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ced9e-c1d0-45c4-bbd5-fc6d67652cfb admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.841597 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ced9e-c1d0-45c4-bbd5-fc6d67652cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.841871 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ced9e-c1d0-45c4-bbd5-fc6d67652cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.842668 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ced9e-c1d0-45c4-bbd5-fc6d67652cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.844250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ced9e-c1d0-45c4-bbd5-fc6d67652cfb admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.846317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:33.852641 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd751a-eb9a-4dd6-b61b-5df970e401b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d773fbf1acca4df9947f053573486931, user_id=a1d7a67f595b44a89abe80bc4f714821, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.857375 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd751a-eb9a-4dd6-b61b-5df970e401b7 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.859192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 804/1582] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:33.859957 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ced9e-c1d0-45c4-bbd5-fc6d67652cfb admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.873220 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ced9e-c1d0-45c4-bbd5-fc6d67652cfb admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.875601 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 779/1583] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:33 2022] PUT /identity/v3/projects/d773fbf1acca4df9947f053573486931/users/a1d7a67f595b44a89abe80bc4f714821/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:33.875877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c72fc5-86a3-4539-afe6-59429ce0e40a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.876187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c72fc5-86a3-4539-afe6-59429ce0e40a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.876491 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c72fc5-86a3-4539-afe6-59429ce0e40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.884010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:33.907774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c1c72fc5-86a3-4539-afe6-59429ce0e40a None None] MFA Rules not processed for user `46c447b35cd14422baaf480f22dc1ba3`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:33.909005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c1c72fc5-86a3-4539-afe6-59429ce0e40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:33.946758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dcb83-486d-403c-ad94-a59d287bcc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.947409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dcb83-486d-403c-ad94-a59d287bcc34 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.947681 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dcb83-486d-403c-ad94-a59d287bcc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.947943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dcb83-486d-403c-ad94-a59d287bcc34 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.948601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0dcb83-486d-403c-ad94-a59d287bcc34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.950591 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0dcb83-486d-403c-ad94-a59d287bcc34 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.978782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 805/1584] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:33.979505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dcb83-486d-403c-ad94-a59d287bcc34 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.985587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 780/1585] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:33.987086 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c72fc5-86a3-4539-afe6-59429ce0e40a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.992622 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dcb83-486d-403c-ad94-a59d287bcc34 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.997469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5aeee7-733f-4363-b090-e18cc4a5a0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.998573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5aeee7-733f-4363-b090-e18cc4a5a0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.998877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5aeee7-733f-4363-b090-e18cc4a5a0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.999144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5aeee7-733f-4363-b090-e18cc4a5a0a1 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.999813 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5aeee7-733f-4363-b090-e18cc4a5a0a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.999944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c72fc5-86a3-4539-afe6-59429ce0e40a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.001745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5aeee7-733f-4363-b090-e18cc4a5a0a1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.005235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b57f5-4434-405d-9a5e-48e0557e52ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.005913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b57f5-4434-405d-9a5e-48e0557e52ea admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.006188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b57f5-4434-405d-9a5e-48e0557e52ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.006469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b57f5-4434-405d-9a5e-48e0557e52ea admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.007156 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b57f5-4434-405d-9a5e-48e0557e52ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.009217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b57f5-4434-405d-9a5e-48e0557e52ea admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.034151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 806/1586] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/users => generated 295 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:34.034151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5aeee7-733f-4363-b090-e18cc4a5a0a1 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.036763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 781/1587] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/users => generated 291 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:34.036925 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b57f5-4434-405d-9a5e-48e0557e52ea admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.047373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5aeee7-733f-4363-b090-e18cc4a5a0a1 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.047647 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b57f5-4434-405d-9a5e-48e0557e52ea admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.052319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51115a83-9599-4945-b17f-9c1f118665aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.052930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51115a83-9599-4945-b17f-9c1f118665aa admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.053209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51115a83-9599-4945-b17f-9c1f118665aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.053478 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51115a83-9599-4945-b17f-9c1f118665aa admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.053894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d0844-f070-4a88-8b89-d1c32086e142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.054281 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51115a83-9599-4945-b17f-9c1f118665aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.054505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d0844-f070-4a88-8b89-d1c32086e142 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.054807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d0844-f070-4a88-8b89-d1c32086e142 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.055075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d0844-f070-4a88-8b89-d1c32086e142 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.055768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d0844-f070-4a88-8b89-d1c32086e142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.056244 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51115a83-9599-4945-b17f-9c1f118665aa admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.057631 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d0844-f070-4a88-8b89-d1c32086e142 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.071665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 807/1588] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:34.072513 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d0844-f070-4a88-8b89-d1c32086e142 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.083221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 782/1589] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:34 2022] POST /identity/v3/users => generated 296 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:34.083441 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d0844-f070-4a88-8b89-d1c32086e142 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.084283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0d09f-bfce-42ec-9aed-982a0d4db503 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.084570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0d09f-bfce-42ec-9aed-982a0d4db503 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.084872 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0d09f-bfce-42ec-9aed-982a0d4db503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.090347 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a73b9-e009-4bd6-8830-1e68d77e9126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.091253 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a73b9-e009-4bd6-8830-1e68d77e9126 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.091533 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a73b9-e009-4bd6-8830-1e68d77e9126 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.091810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a73b9-e009-4bd6-8830-1e68d77e9126 admin admin] PATH_INFO: `/v3/projects/7f9fbd8d38b743e9915a370d8f1d77c4/users/a357a3d27e4e41adae1c56ee39ae5aa7/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.104912 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-7ed0d09f-bfce-42ec-9aed-982a0d4db503 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:34.106686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7ed0d09f-bfce-42ec-9aed-982a0d4db503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:34.109822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a73b9-e009-4bd6-8830-1e68d77e9126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f9fbd8d38b743e9915a370d8f1d77c4, user_id=a357a3d27e4e41adae1c56ee39ae5aa7, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.111123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: WARNING py.warnings [None req-053a73b9-e009-4bd6-8830-1e68d77e9126 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:49:34.111123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: warnings.warn(msg) Aug 05 19:49:34.111123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: Aug 05 19:49:34.113276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a73b9-e009-4bd6-8830-1e68d77e9126 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.118326 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 783/1590] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:34.118326 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0d09f-bfce-42ec-9aed-982a0d4db503 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.128681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0d09f-bfce-42ec-9aed-982a0d4db503 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.134394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77876f-a3df-4985-8055-600067d9c403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.135068 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77876f-a3df-4985-8055-600067d9c403 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.135362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77876f-a3df-4985-8055-600067d9c403 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.135635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77876f-a3df-4985-8055-600067d9c403 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.136384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77876f-a3df-4985-8055-600067d9c403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.138004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77876f-a3df-4985-8055-600067d9c403 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.140708 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 808/1591] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:34 2022] PUT /identity/v3/projects/7f9fbd8d38b743e9915a370d8f1d77c4/users/a357a3d27e4e41adae1c56ee39ae5aa7/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:34.141806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a73b9-e009-4bd6-8830-1e68d77e9126 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.149523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 784/1592] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:34.150502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77876f-a3df-4985-8055-600067d9c403 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.160517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77876f-a3df-4985-8055-600067d9c403 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.212742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a73b9-e009-4bd6-8830-1e68d77e9126 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.218012 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8fe9a-522c-4e9f-a096-ac1acc3d5ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.218743 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8fe9a-522c-4e9f-a096-ac1acc3d5ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.218980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8fe9a-522c-4e9f-a096-ac1acc3d5ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.219218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8fe9a-522c-4e9f-a096-ac1acc3d5ea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.219918 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8fe9a-522c-4e9f-a096-ac1acc3d5ea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.221872 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8fe9a-522c-4e9f-a096-ac1acc3d5ea2 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.222306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c0d6d-5fe9-4a9c-a24b-f6cc345b9ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.222958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c0d6d-5fe9-4a9c-a24b-f6cc345b9ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.223238 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c0d6d-5fe9-4a9c-a24b-f6cc345b9ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.223513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c0d6d-5fe9-4a9c-a24b-f6cc345b9ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.224217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c0d6d-5fe9-4a9c-a24b-f6cc345b9ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.225844 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c0d6d-5fe9-4a9c-a24b-f6cc345b9ce8 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.237772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 785/1593] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:34.238358 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c0d6d-5fe9-4a9c-a24b-f6cc345b9ce8 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.253163 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c0d6d-5fe9-4a9c-a24b-f6cc345b9ce8 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.253508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 809/1594] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:34 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:34.254484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8fe9a-522c-4e9f-a096-ac1acc3d5ea2 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.276987 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:34.294279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:34.326959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d070ec-3367-4e2c-a5b7-11a4f63f4974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.328221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d070ec-3367-4e2c-a5b7-11a4f63f4974 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.328507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d070ec-3367-4e2c-a5b7-11a4f63f4974 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.328808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d070ec-3367-4e2c-a5b7-11a4f63f4974 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.329325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8fe9a-522c-4e9f-a096-ac1acc3d5ea2 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.329667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d070ec-3367-4e2c-a5b7-11a4f63f4974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.331313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d070ec-3367-4e2c-a5b7-11a4f63f4974 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.334280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa8820-a3ab-4020-a8fb-28ae3fb7345e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.334934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa8820-a3ab-4020-a8fb-28ae3fb7345e admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.335211 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa8820-a3ab-4020-a8fb-28ae3fb7345e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.335479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa8820-a3ab-4020-a8fb-28ae3fb7345e admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.336179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa8820-a3ab-4020-a8fb-28ae3fb7345e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.337782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa8820-a3ab-4020-a8fb-28ae3fb7345e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.342961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 786/1595] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:34.344085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d070ec-3367-4e2c-a5b7-11a4f63f4974 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.351016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 810/1596] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:34.351868 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa8820-a3ab-4020-a8fb-28ae3fb7345e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.356372 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d070ec-3367-4e2c-a5b7-11a4f63f4974 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.362693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a7a00-5ef2-4c42-bf09-274df4b90496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.363359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a7a00-5ef2-4c42-bf09-274df4b90496 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.363683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a7a00-5ef2-4c42-bf09-274df4b90496 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.363957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a7a00-5ef2-4c42-bf09-274df4b90496 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.364669 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a7a00-5ef2-4c42-bf09-274df4b90496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.366266 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a7a00-5ef2-4c42-bf09-274df4b90496 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.381243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 787/1597] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:34.381243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb0e29-4e79-4c39-98db-a696ae1e05ee None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.381243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb0e29-4e79-4c39-98db-a696ae1e05ee None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.381243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb0e29-4e79-4c39-98db-a696ae1e05ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.403975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-d0cb0e29-4e79-4c39-98db-a696ae1e05ee None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:34.405502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d0cb0e29-4e79-4c39-98db-a696ae1e05ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:34.413765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 788/1598] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:34.415494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb0e29-4e79-4c39-98db-a696ae1e05ee None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.427387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa8820-a3ab-4020-a8fb-28ae3fb7345e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.429879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb0e29-4e79-4c39-98db-a696ae1e05ee None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.431156 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243d8e7-cd1b-4199-bcc1-1994416c9ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.431780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243d8e7-cd1b-4199-bcc1-1994416c9ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.432314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243d8e7-cd1b-4199-bcc1-1994416c9ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.432586 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243d8e7-cd1b-4199-bcc1-1994416c9ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.433698 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7890510-87de-4a2d-8f86-f7267e16650f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.436225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7890510-87de-4a2d-8f86-f7267e16650f admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.436507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7890510-87de-4a2d-8f86-f7267e16650f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.436779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7890510-87de-4a2d-8f86-f7267e16650f admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.437485 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7890510-87de-4a2d-8f86-f7267e16650f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.439127 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7890510-87de-4a2d-8f86-f7267e16650f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.444065 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243d8e7-cd1b-4199-bcc1-1994416c9ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.445527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243d8e7-cd1b-4199-bcc1-1994416c9ac2 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.451332 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 789/1599] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:34.453326 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7890510-87de-4a2d-8f86-f7267e16650f admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.458880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 811/1600] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:34.459715 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243d8e7-cd1b-4199-bcc1-1994416c9ac2 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.466811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7890510-87de-4a2d-8f86-f7267e16650f admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.470676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243d8e7-cd1b-4199-bcc1-1994416c9ac2 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.472548 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d7932-7c5f-42b5-a6ce-6eaa29fa0936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.473180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d7932-7c5f-42b5-a6ce-6eaa29fa0936 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.473458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d7932-7c5f-42b5-a6ce-6eaa29fa0936 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.473755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d7932-7c5f-42b5-a6ce-6eaa29fa0936 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.474717 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d7932-7c5f-42b5-a6ce-6eaa29fa0936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.478792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d7932-7c5f-42b5-a6ce-6eaa29fa0936 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.480773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bd8ce-fef6-4962-96dd-cf6cf5ea4453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.482340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bd8ce-fef6-4962-96dd-cf6cf5ea4453 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.482340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bd8ce-fef6-4962-96dd-cf6cf5ea4453 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.482340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bd8ce-fef6-4962-96dd-cf6cf5ea4453 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.482820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bd8ce-fef6-4962-96dd-cf6cf5ea4453 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.485579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bd8ce-fef6-4962-96dd-cf6cf5ea4453 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.489608 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 790/1601] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:34.490613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d7932-7c5f-42b5-a6ce-6eaa29fa0936 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.502455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d7932-7c5f-42b5-a6ce-6eaa29fa0936 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.507730 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5f655-85e4-45ac-bb16-361576b691fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.508405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5f655-85e4-45ac-bb16-361576b691fa admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.508678 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5f655-85e4-45ac-bb16-361576b691fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.508948 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5f655-85e4-45ac-bb16-361576b691fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.509649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5f655-85e4-45ac-bb16-361576b691fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.511360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5f655-85e4-45ac-bb16-361576b691fa admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.519083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 812/1602] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:34 2022] POST /identity/v3/users => generated 292 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:34.520301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bd8ce-fef6-4962-96dd-cf6cf5ea4453 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.528710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 791/1603] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:34.529445 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5f655-85e4-45ac-bb16-361576b691fa admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.534721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bd8ce-fef6-4962-96dd-cf6cf5ea4453 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.540405 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77740b0-5695-4b28-9921-40e45919c2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.541112 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77740b0-5695-4b28-9921-40e45919c2a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.541400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77740b0-5695-4b28-9921-40e45919c2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.541544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5f655-85e4-45ac-bb16-361576b691fa admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.541651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77740b0-5695-4b28-9921-40e45919c2a1 admin admin] PATH_INFO: `/v3/projects/c374be6333184ae3900d8c4dd4717101/users/899fd8ee4c614bec8ec709463b3e3b87/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.546963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df98958-a0db-4558-a57f-b2d961fe4ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.547596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df98958-a0db-4558-a57f-b2d961fe4ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.547873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df98958-a0db-4558-a57f-b2d961fe4ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.548196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df98958-a0db-4558-a57f-b2d961fe4ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.548897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df98958-a0db-4558-a57f-b2d961fe4ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.550570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df98958-a0db-4558-a57f-b2d961fe4ef1 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.562156 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 792/1604] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:34.562864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df98958-a0db-4558-a57f-b2d961fe4ef1 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.564091 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77740b0-5695-4b28-9921-40e45919c2a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c374be6333184ae3900d8c4dd4717101, user_id=899fd8ee4c614bec8ec709463b3e3b87, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.567140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77740b0-5695-4b28-9921-40e45919c2a1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.576564 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df98958-a0db-4558-a57f-b2d961fe4ef1 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.584388 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 813/1605] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:34 2022] PUT /identity/v3/projects/c374be6333184ae3900d8c4dd4717101/users/899fd8ee4c614bec8ec709463b3e3b87/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:34.585065 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77740b0-5695-4b28-9921-40e45919c2a1 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.647981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59432351-2500-4279-aff9-70956f763f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.649049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59432351-2500-4279-aff9-70956f763f58 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.649591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59432351-2500-4279-aff9-70956f763f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.650113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59432351-2500-4279-aff9-70956f763f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.667179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77740b0-5695-4b28-9921-40e45919c2a1 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.675358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826c4bf-07aa-4416-a39b-c533d1355c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.676076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826c4bf-07aa-4416-a39b-c533d1355c34 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.677085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826c4bf-07aa-4416-a39b-c533d1355c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.677085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826c4bf-07aa-4416-a39b-c533d1355c34 admin admin] PATH_INFO: `/v3/projects/21db26f543ab4a67b932184ba810f4e9/users/074f3ddb689c4a60a05d3267080d7e7a/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.698864 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826c4bf-07aa-4416-a39b-c533d1355c34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21db26f543ab4a67b932184ba810f4e9, user_id=074f3ddb689c4a60a05d3267080d7e7a, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.702754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826c4bf-07aa-4416-a39b-c533d1355c34 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.729182 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 814/1606] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:34 2022] PUT /identity/v3/projects/21db26f543ab4a67b932184ba810f4e9/users/074f3ddb689c4a60a05d3267080d7e7a/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:34.730125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826c4bf-07aa-4416-a39b-c533d1355c34 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.755227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59432351-2500-4279-aff9-70956f763f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.755575 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:34.757070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59432351-2500-4279-aff9-70956f763f58 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.769435 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 793/1607] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:34.770145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59432351-2500-4279-aff9-70956f763f58 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.784346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59432351-2500-4279-aff9-70956f763f58 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.788535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:34.838667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826c4bf-07aa-4416-a39b-c533d1355c34 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.845211 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330c81c-2298-4a11-b78f-62f195729691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.845994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330c81c-2298-4a11-b78f-62f195729691 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.846347 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330c81c-2298-4a11-b78f-62f195729691 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.846724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330c81c-2298-4a11-b78f-62f195729691 admin admin] PATH_INFO: `/v3/projects/9be735c81c7a41b8a482e8f72a71bdef/users/e8f7435928a74258af5478959bd939bb/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.860793 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c8d19-269f-4415-81db-9893c82be53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.864656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c8d19-269f-4415-81db-9893c82be53b admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.864656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c8d19-269f-4415-81db-9893c82be53b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.864656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c8d19-269f-4415-81db-9893c82be53b admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.864656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c8d19-269f-4415-81db-9893c82be53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.865447 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c8d19-269f-4415-81db-9893c82be53b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.868256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330c81c-2298-4a11-b78f-62f195729691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be735c81c7a41b8a482e8f72a71bdef, user_id=e8f7435928a74258af5478959bd939bb, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.870822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330c81c-2298-4a11-b78f-62f195729691 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.881959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 794/1608] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:34.881959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c8d19-269f-4415-81db-9893c82be53b admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.894710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 815/1609] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:34 2022] PUT /identity/v3/projects/9be735c81c7a41b8a482e8f72a71bdef/users/e8f7435928a74258af5478959bd939bb/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:34.896573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c8d19-269f-4415-81db-9893c82be53b admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.897547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330c81c-2298-4a11-b78f-62f195729691 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.978282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330c81c-2298-4a11-b78f-62f195729691 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.985759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9289d7-6331-4c41-8284-43b68cf02388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.986152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9289d7-6331-4c41-8284-43b68cf02388 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.986458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9289d7-6331-4c41-8284-43b68cf02388 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.986729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9289d7-6331-4c41-8284-43b68cf02388 admin admin] PATH_INFO: `/v3/projects/afed1554d1394cfc8abea0e6c5e15b3f/users/d183fcdae86448caa81c09823dd3884e/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.008341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9289d7-6331-4c41-8284-43b68cf02388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afed1554d1394cfc8abea0e6c5e15b3f, user_id=d183fcdae86448caa81c09823dd3884e, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:35.011291 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9289d7-6331-4c41-8284-43b68cf02388 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:35.019062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627cdaa-556a-46be-b712-3581dbb655eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:35.019953 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627cdaa-556a-46be-b712-3581dbb655eb admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.020298 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627cdaa-556a-46be-b712-3581dbb655eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.020632 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627cdaa-556a-46be-b712-3581dbb655eb admin admin] PATH_INFO: `/v3/projects/664cf2ad8cda4ee089a12d107a785b66/users/6ecba1d2ea2c493c969522f33f33d6de/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.032998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 816/1610] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:34 2022] PUT /identity/v3/projects/afed1554d1394cfc8abea0e6c5e15b3f/users/d183fcdae86448caa81c09823dd3884e/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:35.035978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53b2bd-9454-4950-a312-960eb64f0d74 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.037520 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53b2bd-9454-4950-a312-960eb64f0d74 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.038242 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53b2bd-9454-4950-a312-960eb64f0d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.060755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627cdaa-556a-46be-b712-3581dbb655eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=664cf2ad8cda4ee089a12d107a785b66, user_id=6ecba1d2ea2c493c969522f33f33d6de, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:35.063723 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627cdaa-556a-46be-b712-3581dbb655eb admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:35.088620 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 795/1611] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:34 2022] PUT /identity/v3/projects/664cf2ad8cda4ee089a12d107a785b66/users/6ecba1d2ea2c493c969522f33f33d6de/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:35.089737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e8bd3-0cc4-491e-b7ef-c9bc5ae00456 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.090743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e8bd3-0cc4-491e-b7ef-c9bc5ae00456 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.095131 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e8bd3-0cc4-491e-b7ef-c9bc5ae00456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.095403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-aa53b2bd-9454-4950-a312-960eb64f0d74 None None] MFA Rules not processed for user `899fd8ee4c614bec8ec709463b3e3b87`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:35.096786 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-aa53b2bd-9454-4950-a312-960eb64f0d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:35.127918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-286e8bd3-0cc4-491e-b7ef-c9bc5ae00456 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:35.129247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-286e8bd3-0cc4-491e-b7ef-c9bc5ae00456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:35.208534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 817/1612] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:35.214039 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53b2bd-9454-4950-a312-960eb64f0d74 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:35.227759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:35.247884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 796/1613] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:35.252320 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201653fa-c5cb-4342-afde-9b099f0307e4 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.252761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201653fa-c5cb-4342-afde-9b099f0307e4 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.253091 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201653fa-c5cb-4342-afde-9b099f0307e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.291205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-201653fa-c5cb-4342-afde-9b099f0307e4 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:35.292861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-201653fa-c5cb-4342-afde-9b099f0307e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:35.306863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 797/1614] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:35.306863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233182-8cd8-4883-a11b-0800e081ff77 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.306863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233182-8cd8-4883-a11b-0800e081ff77 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.306863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233182-8cd8-4883-a11b-0800e081ff77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.321054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53b2bd-9454-4950-a312-960eb64f0d74 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:35.332163 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495d562-9833-43f0-865b-6d8656642836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:35.332909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495d562-9833-43f0-865b-6d8656642836 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.333188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495d562-9833-43f0-865b-6d8656642836 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.333456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495d562-9833-43f0-865b-6d8656642836 admin admin] PATH_INFO: `/v3/projects/63eaf9e424af4739a88ff0f3a12ef464/users/d24f9afb647142bfa6b243037d0ce20c/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.339957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-03233182-8cd8-4883-a11b-0800e081ff77 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:35.344542 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-03233182-8cd8-4883-a11b-0800e081ff77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:35.362433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495d562-9833-43f0-865b-6d8656642836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63eaf9e424af4739a88ff0f3a12ef464, user_id=d24f9afb647142bfa6b243037d0ce20c, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:35.365853 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495d562-9833-43f0-865b-6d8656642836 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:35.367672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 798/1615] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:35.370915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94998ff4-0a5a-4a6a-ae76-9dce39327e89 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.372825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94998ff4-0a5a-4a6a-ae76-9dce39327e89 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.373574 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94998ff4-0a5a-4a6a-ae76-9dce39327e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.388968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 818/1616] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:35 2022] PUT /identity/v3/projects/63eaf9e424af4739a88ff0f3a12ef464/users/d24f9afb647142bfa6b243037d0ce20c/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:35.391755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3180a-abd3-4c41-a206-cc9f660d9268 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.392116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3180a-abd3-4c41-a206-cc9f660d9268 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.392931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3180a-abd3-4c41-a206-cc9f660d9268 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.395120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 819/1617] 10.222.0.86 () {54 vars in 962 bytes} [Fri Aug 5 19:49:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:35.396352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3180a-abd3-4c41-a206-cc9f660d9268 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:35.438494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:35.441494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-94998ff4-0a5a-4a6a-ae76-9dce39327e89 None None] MFA Rules not processed for user `6ecba1d2ea2c493c969522f33f33d6de`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:35.443514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-94998ff4-0a5a-4a6a-ae76-9dce39327e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:35.497088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3180a-abd3-4c41-a206-cc9f660d9268 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:35.509490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea761b-1a02-49f6-9319-528b53034498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:35.510251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea761b-1a02-49f6-9319-528b53034498 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.510522 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea761b-1a02-49f6-9319-528b53034498 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.510834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea761b-1a02-49f6-9319-528b53034498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.562614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 799/1618] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:35.562614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94998ff4-0a5a-4a6a-ae76-9dce39327e89 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:35.577335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94998ff4-0a5a-4a6a-ae76-9dce39327e89 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:35.584741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f15ed-6ad9-49b2-9db5-e64c8725d914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:35.585586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f15ed-6ad9-49b2-9db5-e64c8725d914 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.585938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f15ed-6ad9-49b2-9db5-e64c8725d914 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.586307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f15ed-6ad9-49b2-9db5-e64c8725d914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.623153 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea761b-1a02-49f6-9319-528b53034498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:35.624479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea761b-1a02-49f6-9319-528b53034498 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:35.637576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 820/1619] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:35.638855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea761b-1a02-49f6-9319-528b53034498 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:35.640187 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a7ea761b-1a02-49f6-9319-528b53034498 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:35.652451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea761b-1a02-49f6-9319-528b53034498 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:35.657177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba2669-3102-4c21-9a9b-0d7ab737f090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:35.657826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba2669-3102-4c21-9a9b-0d7ab737f090 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.658156 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba2669-3102-4c21-9a9b-0d7ab737f090 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.658433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba2669-3102-4c21-9a9b-0d7ab737f090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.672492 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba2669-3102-4c21-9a9b-0d7ab737f090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:35.674560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba2669-3102-4c21-9a9b-0d7ab737f090 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:35.689165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 821/1620] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:35.689674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f15ed-6ad9-49b2-9db5-e64c8725d914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:35.690503 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba2669-3102-4c21-9a9b-0d7ab737f090 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:35.692128 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f15ed-6ad9-49b2-9db5-e64c8725d914 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:35.705097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba2669-3102-4c21-9a9b-0d7ab737f090 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:35.708077 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 800/1621] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:35.709924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae7b0f-98c7-46df-b79f-9ca9362e166a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.710370 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae7b0f-98c7-46df-b79f-9ca9362e166a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.711088 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae7b0f-98c7-46df-b79f-9ca9362e166a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.712366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1c889-51e5-4bcc-aa0b-1facc898c59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:35.712756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1c889-51e5-4bcc-aa0b-1facc898c59e admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.713236 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1c889-51e5-4bcc-aa0b-1facc898c59e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.713535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1c889-51e5-4bcc-aa0b-1facc898c59e admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.714575 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1c889-51e5-4bcc-aa0b-1facc898c59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:35.716590 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1c889-51e5-4bcc-aa0b-1facc898c59e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:35.738231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 822/1622] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:35.739245 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1c889-51e5-4bcc-aa0b-1facc898c59e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:35.753809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1c889-51e5-4bcc-aa0b-1facc898c59e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:35.759821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020633cd-6787-433f-bba4-7c2e93213869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:35.760717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020633cd-6787-433f-bba4-7c2e93213869 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:35.761294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020633cd-6787-433f-bba4-7c2e93213869 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:35.762619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020633cd-6787-433f-bba4-7c2e93213869 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:35.763904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020633cd-6787-433f-bba4-7c2e93213869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:35.766950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020633cd-6787-433f-bba4-7c2e93213869 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:35.791449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-94ae7b0f-98c7-46df-b79f-9ca9362e166a None None] MFA Rules not processed for user `ab8f9d9cf4ca4e38ae6eb67e949b40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:35.792726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-94ae7b0f-98c7-46df-b79f-9ca9362e166a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:35.809470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:35.835013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 823/1623] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:35 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:35.839366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020633cd-6787-433f-bba4-7c2e93213869 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:35.894123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 801/1624] 10.222.0.86 () {58 vars in 1059 bytes} [Fri Aug 5 19:49:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:35.895233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae7b0f-98c7-46df-b79f-9ca9362e166a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:35.912372 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae7b0f-98c7-46df-b79f-9ca9362e166a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:35.934521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.000566 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b1008-2fe9-42c0-9b17-080619351abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.001894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b1008-2fe9-42c0-9b17-080619351abe admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.002184 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b1008-2fe9-42c0-9b17-080619351abe admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.002671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b1008-2fe9-42c0-9b17-080619351abe admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.003498 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b1008-2fe9-42c0-9b17-080619351abe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.006313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b1008-2fe9-42c0-9b17-080619351abe admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.014502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020633cd-6787-433f-bba4-7c2e93213869 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.026152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a701776-65b3-41fb-9e39-c1f8b382bc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.028819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a701776-65b3-41fb-9e39-c1f8b382bc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.029170 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a701776-65b3-41fb-9e39-c1f8b382bc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.031199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a701776-65b3-41fb-9e39-c1f8b382bc9b admin admin] PATH_INFO: `/v3/projects/63eaf9e424af4739a88ff0f3a12ef464/users/d24f9afb647142bfa6b243037d0ce20c/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.042558 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a701776-65b3-41fb-9e39-c1f8b382bc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=63eaf9e424af4739a88ff0f3a12ef464, user_id=d24f9afb647142bfa6b243037d0ce20c, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.056482 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a701776-65b3-41fb-9e39-c1f8b382bc9b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.057361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 802/1625] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:35 2022] POST /identity/v3/users => generated 295 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:36.058246 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b1008-2fe9-42c0-9b17-080619351abe admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.084774 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 824/1626] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:35 2022] PUT /identity/v3/projects/63eaf9e424af4739a88ff0f3a12ef464/users/d24f9afb647142bfa6b243037d0ce20c/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:36.095012 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a701776-65b3-41fb-9e39-c1f8b382bc9b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.238886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b1008-2fe9-42c0-9b17-080619351abe admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.244729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a701776-65b3-41fb-9e39-c1f8b382bc9b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.260124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b915f4b-7611-4f36-a329-91ec244f3a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.260820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b915f4b-7611-4f36-a329-91ec244f3a78 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.261101 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b915f4b-7611-4f36-a329-91ec244f3a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.261372 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b915f4b-7611-4f36-a329-91ec244f3a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.264412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b915f4b-7611-4f36-a329-91ec244f3a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.266201 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b915f4b-7611-4f36-a329-91ec244f3a78 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.298535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 825/1627] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:36.299573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ccaf8-0e4d-4703-9b65-94e9265475b4 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.299859 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ccaf8-0e4d-4703-9b65-94e9265475b4 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.300193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ccaf8-0e4d-4703-9b65-94e9265475b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.342959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-917ccaf8-0e4d-4703-9b65-94e9265475b4 None None] MFA Rules not processed for user `d24f9afb647142bfa6b243037d0ce20c`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:36.344114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-917ccaf8-0e4d-4703-9b65-94e9265475b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:36.349919 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079203e-6a0f-4c2d-ad80-7e0d935a7626 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.351367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079203e-6a0f-4c2d-ad80-7e0d935a7626 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.351808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079203e-6a0f-4c2d-ad80-7e0d935a7626 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.352797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079203e-6a0f-4c2d-ad80-7e0d935a7626 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.420329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.437658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 826/1628] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:36.438941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ccaf8-0e4d-4703-9b65-94e9265475b4 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.463752 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079203e-6a0f-4c2d-ad80-7e0d935a7626 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.468073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079203e-6a0f-4c2d-ad80-7e0d935a7626 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.472535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ccaf8-0e4d-4703-9b65-94e9265475b4 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.492932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 803/1629] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:36.495817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00aea3-81a1-4b36-ae93-d084d8cb1542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.497425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00aea3-81a1-4b36-ae93-d084d8cb1542 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.497852 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00aea3-81a1-4b36-ae93-d084d8cb1542 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.498262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00aea3-81a1-4b36-ae93-d084d8cb1542 admin admin] PATH_INFO: `/v3/projects/507ce3e3dce6428bbf81c1f4b550a5fc/users/867cdfbc077940dd8947140a7416c786/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.501011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079203e-6a0f-4c2d-ad80-7e0d935a7626 service nova] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.519450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.549730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00aea3-81a1-4b36-ae93-d084d8cb1542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=507ce3e3dce6428bbf81c1f4b550a5fc, user_id=867cdfbc077940dd8947140a7416c786, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.552627 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00aea3-81a1-4b36-ae93-d084d8cb1542 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.591997 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 827/1630] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:36 2022] PUT /identity/v3/projects/507ce3e3dce6428bbf81c1f4b550a5fc/users/867cdfbc077940dd8947140a7416c786/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:36.595110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00aea3-81a1-4b36-ae93-d084d8cb1542 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.676673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079203e-6a0f-4c2d-ad80-7e0d935a7626 service nova] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.812961 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00aea3-81a1-4b36-ae93-d084d8cb1542 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.827632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15de9e-7a07-40a5-9bdb-043ffa1daac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.829766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15de9e-7a07-40a5-9bdb-043ffa1daac5 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.832951 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15de9e-7a07-40a5-9bdb-043ffa1daac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.833582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15de9e-7a07-40a5-9bdb-043ffa1daac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.836271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15de9e-7a07-40a5-9bdb-043ffa1daac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.841027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df15de9e-7a07-40a5-9bdb-043ffa1daac5 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.847813 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ac18d-4b74-4c25-9f58-ee86d13e6014 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.849875 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ac18d-4b74-4c25-9f58-ee86d13e6014 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.850842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ac18d-4b74-4c25-9f58-ee86d13e6014 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.852925 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ac18d-4b74-4c25-9f58-ee86d13e6014 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.885357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 828/1631] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:36.885357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15de9e-7a07-40a5-9bdb-043ffa1daac5 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.921258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df15de9e-7a07-40a5-9bdb-043ffa1daac5 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.937349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b5f71-c262-4c37-8794-905d4d90fdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.938790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b5f71-c262-4c37-8794-905d4d90fdef admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.939608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b5f71-c262-4c37-8794-905d4d90fdef admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.940042 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b5f71-c262-4c37-8794-905d4d90fdef admin admin] PATH_INFO: `/v3/projects/507ce3e3dce6428bbf81c1f4b550a5fc/users/867cdfbc077940dd8947140a7416c786/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.944656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b5f71-c262-4c37-8794-905d4d90fdef admin admin] RBAC: Authorizing `identity:create_grant(project_id=507ce3e3dce6428bbf81c1f4b550a5fc, user_id=867cdfbc077940dd8947140a7416c786, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.953546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b5f71-c262-4c37-8794-905d4d90fdef admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.037068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 829/1632] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:36 2022] PUT /identity/v3/projects/507ce3e3dce6428bbf81c1f4b550a5fc/users/867cdfbc077940dd8947140a7416c786/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:37.040509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e137f-19ca-4dff-aaa4-31e219797cfb None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.040823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e137f-19ca-4dff-aaa4-31e219797cfb None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.041960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e137f-19ca-4dff-aaa4-31e219797cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.053546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ac18d-4b74-4c25-9f58-ee86d13e6014 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.055962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ac18d-4b74-4c25-9f58-ee86d13e6014 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.081284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 804/1633] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:37.161182 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-b59e137f-19ca-4dff-aaa4-31e219797cfb None None] MFA Rules not processed for user `867cdfbc077940dd8947140a7416c786`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:37.165591 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b59e137f-19ca-4dff-aaa4-31e219797cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:37.284500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ac18d-4b74-4c25-9f58-ee86d13e6014 service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.302354 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ac18d-4b74-4c25-9f58-ee86d13e6014 service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.370113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 830/1634] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:37.370113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b310f-c49b-4a06-82f7-96bb8f98b595 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.370113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b310f-c49b-4a06-82f7-96bb8f98b595 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.370113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b310f-c49b-4a06-82f7-96bb8f98b595 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.370113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 831/1635] 10.222.0.86 () {54 vars in 962 bytes} [Fri Aug 5 19:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:37.389676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b310f-c49b-4a06-82f7-96bb8f98b595 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.474932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.474932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45149a3a-948b-4703-b3b4-0140e7a31a46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.474932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45149a3a-948b-4703-b3b4-0140e7a31a46 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.474932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45149a3a-948b-4703-b3b4-0140e7a31a46 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.477897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45149a3a-948b-4703-b3b4-0140e7a31a46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.556486 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b310f-c49b-4a06-82f7-96bb8f98b595 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.580650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc1961-d1fd-45f6-a6db-2a6429658858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.581878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc1961-d1fd-45f6-a6db-2a6429658858 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.582350 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc1961-d1fd-45f6-a6db-2a6429658858 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.582523 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc1961-d1fd-45f6-a6db-2a6429658858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.598838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc1961-d1fd-45f6-a6db-2a6429658858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.600690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc1961-d1fd-45f6-a6db-2a6429658858 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.605028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.623249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 832/1636] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:37.624589 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3cf00-f452-48d1-8e39-61743c75d909 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.629001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3cf00-f452-48d1-8e39-61743c75d909 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.629477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3cf00-f452-48d1-8e39-61743c75d909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.636134 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45149a3a-948b-4703-b3b4-0140e7a31a46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.637811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45149a3a-948b-4703-b3b4-0140e7a31a46 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.658984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 805/1637] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:37.682742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-5ea3cf00-f452-48d1-8e39-61743c75d909 None None] MFA Rules not processed for user `ab8f9d9cf4ca4e38ae6eb67e949b40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:37.691772 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5ea3cf00-f452-48d1-8e39-61743c75d909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:37.794206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45149a3a-948b-4703-b3b4-0140e7a31a46 service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.810214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45149a3a-948b-4703-b3b4-0140e7a31a46 service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.820175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2c65e-4803-4325-86a9-1d4bbffd292d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.823029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2c65e-4803-4325-86a9-1d4bbffd292d service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.825099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2c65e-4803-4325-86a9-1d4bbffd292d service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.825389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2c65e-4803-4325-86a9-1d4bbffd292d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.838020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 833/1638] 10.222.0.86 () {58 vars in 1059 bytes} [Fri Aug 5 19:49:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:37.845975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ad1c6-8a8f-4067-8bfd-99bd5ddd9aad None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.846707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ad1c6-8a8f-4067-8bfd-99bd5ddd9aad None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.847249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ad1c6-8a8f-4067-8bfd-99bd5ddd9aad None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.857116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 834/1639] 10.222.0.86 () {54 vars in 912 bytes} [Fri Aug 5 19:49:37 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:37.861568 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf843d-7fc4-4d40-b3e5-cc05f9056621 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.861860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf843d-7fc4-4d40-b3e5-cc05f9056621 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.862138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf843d-7fc4-4d40-b3e5-cc05f9056621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.924195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-6bdf843d-7fc4-4d40-b3e5-cc05f9056621 None None] MFA Rules not processed for user `ab8f9d9cf4ca4e38ae6eb67e949b40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:37.925890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6bdf843d-7fc4-4d40-b3e5-cc05f9056621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:37.946697 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 835/1640] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:49:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:37.990533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2c65e-4803-4325-86a9-1d4bbffd292d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.993059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2c65e-4803-4325-86a9-1d4bbffd292d service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.016504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 806/1641] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:49:38.107804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf843d-7fc4-4d40-b3e5-cc05f9056621 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.128075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf843d-7fc4-4d40-b3e5-cc05f9056621 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.140771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725ee78-8503-46a5-9d09-60b53aef4f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.141458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725ee78-8503-46a5-9d09-60b53aef4f6e service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.141735 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725ee78-8503-46a5-9d09-60b53aef4f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.142007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725ee78-8503-46a5-9d09-60b53aef4f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.180475 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725ee78-8503-46a5-9d09-60b53aef4f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.182322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725ee78-8503-46a5-9d09-60b53aef4f6e service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.245827 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 836/1642] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:38.629794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bc647-dbc4-4e4c-b702-523821d24efa None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.631071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bc647-dbc4-4e4c-b702-523821d24efa None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.631770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bc647-dbc4-4e4c-b702-523821d24efa None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.634178 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 807/1643] 10.222.0.86 () {54 vars in 912 bytes} [Fri Aug 5 19:49:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:38.647531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48c631-ace9-4300-b2ca-e036fb8e3d01 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.648057 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48c631-ace9-4300-b2ca-e036fb8e3d01 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.648215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48c631-ace9-4300-b2ca-e036fb8e3d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.722398 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-7b48c631-ace9-4300-b2ca-e036fb8e3d01 None None] MFA Rules not processed for user `ab8f9d9cf4ca4e38ae6eb67e949b40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:38.727569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7b48c631-ace9-4300-b2ca-e036fb8e3d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:38.753239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 837/1644] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:49:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:38.770360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bc647-dbc4-4e4c-b702-523821d24efa None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.791652 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bc647-dbc4-4e4c-b702-523821d24efa None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.796100 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe222afa-e5aa-4ddf-a115-c3c8e79c9dc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.797104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe222afa-e5aa-4ddf-a115-c3c8e79c9dc5 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.797473 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe222afa-e5aa-4ddf-a115-c3c8e79c9dc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.797804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe222afa-e5aa-4ddf-a115-c3c8e79c9dc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.814125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe222afa-e5aa-4ddf-a115-c3c8e79c9dc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.814125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe222afa-e5aa-4ddf-a115-c3c8e79c9dc5 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.832554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 808/1645] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:39.052446 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48c631-ace9-4300-b2ca-e036fb8e3d01 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.259104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48c631-ace9-4300-b2ca-e036fb8e3d01 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.270528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.271551 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.272713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.273111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] PATH_INFO: `/v3/users/899fd8ee4c614bec8ec709463b3e3b87` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.276694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=899fd8ee4c614bec8ec709463b3e3b87)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.278481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: WARNING py.warnings [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:49:39.278481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: warnings.warn(msg) Aug 05 19:49:39.278481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: Aug 05 19:49:39.287890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.500614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '899fd8ee4c614bec8ec709463b3e3b87'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:39.553499 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '899fd8ee4c614bec8ec709463b3e3b87'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:39.560501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.573719 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '899fd8ee4c614bec8ec709463b3e3b87'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:39.574582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '899fd8ee4c614bec8ec709463b3e3b87'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:39.575541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '899fd8ee4c614bec8ec709463b3e3b87'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:39.600068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 838/1646] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:49:39 2022] DELETE /identity/v3/users/899fd8ee4c614bec8ec709463b3e3b87 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:39.605663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe86b01-8167-4f0c-89a1-8dd615864951 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.606160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe86b01-8167-4f0c-89a1-8dd615864951 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.606529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe86b01-8167-4f0c-89a1-8dd615864951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.634407 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-8fe86b01-8167-4f0c-89a1-8dd615864951 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:39.635833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-8fe86b01-8167-4f0c-89a1-8dd615864951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:39.732523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 809/1647] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:39.758180 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.760008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:39.850366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f9576-df80-4105-b883-944264ad2bf3 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.856481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b1f12-8360-4af9-bd67-3c24c1484e85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.857589 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b1f12-8360-4af9-bd67-3c24c1484e85 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.857881 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b1f12-8360-4af9-bd67-3c24c1484e85 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.858259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b1f12-8360-4af9-bd67-3c24c1484e85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.880060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b1f12-8360-4af9-bd67-3c24c1484e85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.881904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b1f12-8360-4af9-bd67-3c24c1484e85 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.896992 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 839/1648] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:41.922122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe86b01-8167-4f0c-89a1-8dd615864951 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:41.927315 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-8fe86b01-8167-4f0c-89a1-8dd615864951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:41.966351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe86b01-8167-4f0c-89a1-8dd615864951 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:41.981457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec2544-0292-42bb-9954-7cbf68a82ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:41.983900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec2544-0292-42bb-9954-7cbf68a82ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.986175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec2544-0292-42bb-9954-7cbf68a82ead admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.986890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec2544-0292-42bb-9954-7cbf68a82ead admin admin] PATH_INFO: `/v3/projects/c374be6333184ae3900d8c4dd4717101` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.991448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec2544-0292-42bb-9954-7cbf68a82ead admin admin] RBAC: Authorizing `identity:delete_project(project_id=c374be6333184ae3900d8c4dd4717101)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:41.997335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: WARNING py.warnings [None req-9eec2544-0292-42bb-9954-7cbf68a82ead admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 05 19:49:41.997335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: warnings.warn(msg) Aug 05 19:49:41.997335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: Aug 05 19:49:41.999159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec2544-0292-42bb-9954-7cbf68a82ead admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.092568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:42.165179 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9eec2544-0292-42bb-9954-7cbf68a82ead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c374be6333184ae3900d8c4dd4717101', 'request_id': 'req-9eec2544-0292-42bb-9954-7cbf68a82ead'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:42.208911 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9eec2544-0292-42bb-9954-7cbf68a82ead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c374be6333184ae3900d8c4dd4717101', 'request_id': 'req-9eec2544-0292-42bb-9954-7cbf68a82ead'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:42.237865 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9eec2544-0292-42bb-9954-7cbf68a82ead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c374be6333184ae3900d8c4dd4717101', 'request_id': 'req-9eec2544-0292-42bb-9954-7cbf68a82ead'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:42.267570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9eec2544-0292-42bb-9954-7cbf68a82ead admin admin] The token cache is being invalidate because project c374be6333184ae3900d8c4dd4717101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:49:42.269198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9eec2544-0292-42bb-9954-7cbf68a82ead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:42.271734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9eec2544-0292-42bb-9954-7cbf68a82ead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:42.283200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 810/1649] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:49:41 2022] DELETE /identity/v3/projects/c374be6333184ae3900d8c4dd4717101 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:42.285039 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf626fdf-bab1-4743-b7c5-89990db17d0c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.285885 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf626fdf-bab1-4743-b7c5-89990db17d0c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.286168 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf626fdf-bab1-4743-b7c5-89990db17d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.334363 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-bf626fdf-bab1-4743-b7c5-89990db17d0c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:42.335712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-bf626fdf-bab1-4743-b7c5-89990db17d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:42.501192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 840/1650] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:42.509121 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487d470-f71f-486f-94f0-275e36e23855 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.509544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487d470-f71f-486f-94f0-275e36e23855 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.510021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487d470-f71f-486f-94f0-275e36e23855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.549736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-7487d470-f71f-486f-94f0-275e36e23855 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:42.551495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7487d470-f71f-486f-94f0-275e36e23855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:42.563072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 811/1651] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:42.567646 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf626fdf-bab1-4743-b7c5-89990db17d0c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.581526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf626fdf-bab1-4743-b7c5-89990db17d0c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.593556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256defe8-8e1b-4cd0-9d3c-efe6814f0c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.595825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256defe8-8e1b-4cd0-9d3c-efe6814f0c66 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.597713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256defe8-8e1b-4cd0-9d3c-efe6814f0c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.598545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256defe8-8e1b-4cd0-9d3c-efe6814f0c66 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.601031 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256defe8-8e1b-4cd0-9d3c-efe6814f0c66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:42.602075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256defe8-8e1b-4cd0-9d3c-efe6814f0c66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:42.608283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256defe8-8e1b-4cd0-9d3c-efe6814f0c66 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.655925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 841/1652] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:42.657074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487d470-f71f-486f-94f0-275e36e23855 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.674310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487d470-f71f-486f-94f0-275e36e23855 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.690579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e218952-362f-4228-b614-aa53bd10ab4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.691556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e218952-362f-4228-b614-aa53bd10ab4f admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.691672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e218952-362f-4228-b614-aa53bd10ab4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.692380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e218952-362f-4228-b614-aa53bd10ab4f admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.692652 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e218952-362f-4228-b614-aa53bd10ab4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:42.695072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e218952-362f-4228-b614-aa53bd10ab4f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.739802 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 812/1653] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:42 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:42.744462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256defe8-8e1b-4cd0-9d3c-efe6814f0c66 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.767335 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256defe8-8e1b-4cd0-9d3c-efe6814f0c66 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.862248 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4f54f-97b5-4187-86a3-3da9e2d8191b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.863740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4f54f-97b5-4187-86a3-3da9e2d8191b admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.864445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4f54f-97b5-4187-86a3-3da9e2d8191b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.865049 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4f54f-97b5-4187-86a3-3da9e2d8191b admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.866185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4f54f-97b5-4187-86a3-3da9e2d8191b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:42.870197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4f54f-97b5-4187-86a3-3da9e2d8191b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.917653 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 842/1654] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:42 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:42.923120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e218952-362f-4228-b614-aa53bd10ab4f admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.953658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e218952-362f-4228-b614-aa53bd10ab4f admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.971405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b64af5-6ee7-4570-9260-c2526a8e7a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.972403 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b64af5-6ee7-4570-9260-c2526a8e7a13 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.972873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b64af5-6ee7-4570-9260-c2526a8e7a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.973332 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b64af5-6ee7-4570-9260-c2526a8e7a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.975374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b64af5-6ee7-4570-9260-c2526a8e7a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:42.977147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b64af5-6ee7-4570-9260-c2526a8e7a13 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.999045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 813/1655] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:43.003287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4f54f-97b5-4187-86a3-3da9e2d8191b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.009910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:43.022215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4f54f-97b5-4187-86a3-3da9e2d8191b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.029173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bebce-f05c-4441-bcb8-b513e3feca4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.030056 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bebce-f05c-4441-bcb8-b513e3feca4a admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.030475 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bebce-f05c-4441-bcb8-b513e3feca4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.032351 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bebce-f05c-4441-bcb8-b513e3feca4a admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.033260 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bebce-f05c-4441-bcb8-b513e3feca4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.035075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bebce-f05c-4441-bcb8-b513e3feca4a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.061332 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 843/1656] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:43.064132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b64af5-6ee7-4570-9260-c2526a8e7a13 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.081375 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b64af5-6ee7-4570-9260-c2526a8e7a13 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.087383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07093e-c48f-455d-9a67-3955a6f4d0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.087628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07093e-c48f-455d-9a67-3955a6f4d0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.087911 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07093e-c48f-455d-9a67-3955a6f4d0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.088191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07093e-c48f-455d-9a67-3955a6f4d0f4 admin admin] PATH_INFO: `/v3/projects/7aa23f838ee64bfa95c120df84cd7ee9/users/46d95639a156494b897cb4d7405c827a/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.127376 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07093e-c48f-455d-9a67-3955a6f4d0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa23f838ee64bfa95c120df84cd7ee9, user_id=46d95639a156494b897cb4d7405c827a, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.132114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07093e-c48f-455d-9a67-3955a6f4d0f4 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.162285 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 814/1657] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:43 2022] PUT /identity/v3/projects/7aa23f838ee64bfa95c120df84cd7ee9/users/46d95639a156494b897cb4d7405c827a/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:49:43.179100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ac9b5-f66a-481e-bf62-d0c14cbb5d94 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.179494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ac9b5-f66a-481e-bf62-d0c14cbb5d94 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.179875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ac9b5-f66a-481e-bf62-d0c14cbb5d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.220336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-413ac9b5-f66a-481e-bf62-d0c14cbb5d94 None None] MFA Rules not processed for user `46d95639a156494b897cb4d7405c827a`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:43.221967 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-413ac9b5-f66a-481e-bf62-d0c14cbb5d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:43.395791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 844/1658] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:43.401576 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07093e-c48f-455d-9a67-3955a6f4d0f4 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.584220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07093e-c48f-455d-9a67-3955a6f4d0f4 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.591263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631af03-a0d2-478c-98d6-53e046eaa2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.591912 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631af03-a0d2-478c-98d6-53e046eaa2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.592192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631af03-a0d2-478c-98d6-53e046eaa2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.592470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631af03-a0d2-478c-98d6-53e046eaa2a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.593296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631af03-a0d2-478c-98d6-53e046eaa2a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.596407 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631af03-a0d2-478c-98d6-53e046eaa2a9 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.647291 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 815/1659] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:43 2022] POST /identity/v3/projects => generated 373 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:43.651140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ac9b5-f66a-481e-bf62-d0c14cbb5d94 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.667646 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ac9b5-f66a-481e-bf62-d0c14cbb5d94 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.833837 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b87a6-4b0e-4f59-9742-d59584308c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.834648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b87a6-4b0e-4f59-9742-d59584308c3f admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.834972 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b87a6-4b0e-4f59-9742-d59584308c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.844148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b87a6-4b0e-4f59-9742-d59584308c3f admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.845443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b87a6-4b0e-4f59-9742-d59584308c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.849095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b87a6-4b0e-4f59-9742-d59584308c3f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.897674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 845/1660] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:43 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:43.902244 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631af03-a0d2-478c-98d6-53e046eaa2a9 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.924246 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631af03-a0d2-478c-98d6-53e046eaa2a9 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.939918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e83a41-ebcb-4de1-a507-8ff24dc7876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.940737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e83a41-ebcb-4de1-a507-8ff24dc7876b admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.941144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e83a41-ebcb-4de1-a507-8ff24dc7876b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.941523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e83a41-ebcb-4de1-a507-8ff24dc7876b admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.942344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e83a41-ebcb-4de1-a507-8ff24dc7876b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.966088 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e83a41-ebcb-4de1-a507-8ff24dc7876b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.015731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 816/1661] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:44.018281 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b87a6-4b0e-4f59-9742-d59584308c3f admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.048916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b87a6-4b0e-4f59-9742-d59584308c3f admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.076938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314ec0f-d7cf-418d-ae4d-bb95dc57d34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.076938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314ec0f-d7cf-418d-ae4d-bb95dc57d34f admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.082815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314ec0f-d7cf-418d-ae4d-bb95dc57d34f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.083283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314ec0f-d7cf-418d-ae4d-bb95dc57d34f admin admin] PATH_INFO: `/v3/projects/956d17ef27524da19122aba5f6b83e66/users/f3275a034b8c4cc6858cc285d2127626/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.132675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314ec0f-d7cf-418d-ae4d-bb95dc57d34f admin admin] RBAC: Authorizing `identity:create_grant(project_id=956d17ef27524da19122aba5f6b83e66, user_id=f3275a034b8c4cc6858cc285d2127626, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.143040 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314ec0f-d7cf-418d-ae4d-bb95dc57d34f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.170766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 846/1662] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:44 2022] PUT /identity/v3/projects/956d17ef27524da19122aba5f6b83e66/users/f3275a034b8c4cc6858cc285d2127626/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:44.175761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e83a41-ebcb-4de1-a507-8ff24dc7876b admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.193861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e83a41-ebcb-4de1-a507-8ff24dc7876b admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.242443 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.404767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fabe53-e581-428f-b256-76928c601934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.406484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fabe53-e581-428f-b256-76928c601934 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.406484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fabe53-e581-428f-b256-76928c601934 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.406484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fabe53-e581-428f-b256-76928c601934 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.406814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fabe53-e581-428f-b256-76928c601934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.409026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fabe53-e581-428f-b256-76928c601934 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.430532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 817/1663] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:44.435306 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314ec0f-d7cf-418d-ae4d-bb95dc57d34f admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.464752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314ec0f-d7cf-418d-ae4d-bb95dc57d34f admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.473900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7205a6c-97f9-460f-97b8-08aa6f8d884f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.474613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7205a6c-97f9-460f-97b8-08aa6f8d884f admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.475890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7205a6c-97f9-460f-97b8-08aa6f8d884f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.475890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7205a6c-97f9-460f-97b8-08aa6f8d884f admin admin] PATH_INFO: `/v3/projects/956d17ef27524da19122aba5f6b83e66/users/f3275a034b8c4cc6858cc285d2127626/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.479797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7205a6c-97f9-460f-97b8-08aa6f8d884f admin admin] RBAC: Authorizing `identity:create_grant(project_id=956d17ef27524da19122aba5f6b83e66, user_id=f3275a034b8c4cc6858cc285d2127626, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.489205 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7205a6c-97f9-460f-97b8-08aa6f8d884f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.512606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 847/1664] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:44 2022] PUT /identity/v3/projects/956d17ef27524da19122aba5f6b83e66/users/f3275a034b8c4cc6858cc285d2127626/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:44.530649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab55e0-16c7-48db-9c2c-cfebdf12fcb8 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.531523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab55e0-16c7-48db-9c2c-cfebdf12fcb8 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.532083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab55e0-16c7-48db-9c2c-cfebdf12fcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.626133 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-63ab55e0-16c7-48db-9c2c-cfebdf12fcb8 None None] MFA Rules not processed for user `f3275a034b8c4cc6858cc285d2127626`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:44.627410 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-63ab55e0-16c7-48db-9c2c-cfebdf12fcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:44.972830 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 818/1665] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:44.985977 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7205a6c-97f9-460f-97b8-08aa6f8d884f admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.987877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a7205a6c-97f9-460f-97b8-08aa6f8d884f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:45.090505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.166364 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7205a6c-97f9-460f-97b8-08aa6f8d884f admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.179398 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9350f3-7a5f-47bd-9d1a-d1a4774bf696 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.180071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9350f3-7a5f-47bd-9d1a-d1a4774bf696 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.181229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9350f3-7a5f-47bd-9d1a-d1a4774bf696 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.181513 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9350f3-7a5f-47bd-9d1a-d1a4774bf696 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.207611 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9350f3-7a5f-47bd-9d1a-d1a4774bf696 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.212398 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9350f3-7a5f-47bd-9d1a-d1a4774bf696 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.247080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 848/1666] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:46.427393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab55e0-16c7-48db-9c2c-cfebdf12fcb8 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.652682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab55e0-16c7-48db-9c2c-cfebdf12fcb8 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.666178 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.667020 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.667144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.667408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] PATH_INFO: `/v3/users/46d95639a156494b897cb4d7405c827a` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.673161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46d95639a156494b897cb4d7405c827a)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.674420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.812960 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:46.823385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46d95639a156494b897cb4d7405c827a'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:46.848480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46d95639a156494b897cb4d7405c827a'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:46.850761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46d95639a156494b897cb4d7405c827a'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:46.893079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46d95639a156494b897cb4d7405c827a'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:46.916851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46d95639a156494b897cb4d7405c827a'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:46.929224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 819/1667] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:49:46 2022] DELETE /identity/v3/users/46d95639a156494b897cb4d7405c827a => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:49:46.937677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e3979-c8cb-4fb3-aff7-9a6a244f7898 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.938275 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e3979-c8cb-4fb3-aff7-9a6a244f7898 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.946250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e3979-c8cb-4fb3-aff7-9a6a244f7898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.014897 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-fc5e3979-c8cb-4fb3-aff7-9a6a244f7898 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:47.018808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fc5e3979-c8cb-4fb3-aff7-9a6a244f7898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:47.240470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 849/1668] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:47.289205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.291918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:47.536570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b13e6-635e-42d9-b59a-731002f5a769 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.547352 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3507e7-c493-41b8-91ba-d96b1e9ae965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.548003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3507e7-c493-41b8-91ba-d96b1e9ae965 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.548280 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3507e7-c493-41b8-91ba-d96b1e9ae965 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.548691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3507e7-c493-41b8-91ba-d96b1e9ae965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.589025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3507e7-c493-41b8-91ba-d96b1e9ae965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.597393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3507e7-c493-41b8-91ba-d96b1e9ae965 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.632367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 820/1669] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:49.619441 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e3979-c8cb-4fb3-aff7-9a6a244f7898 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:49.621894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fc5e3979-c8cb-4fb3-aff7-9a6a244f7898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:49.658170 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e3979-c8cb-4fb3-aff7-9a6a244f7898 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.672124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2338d6f-08a4-4111-9a07-963a97b3fff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.675405 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2338d6f-08a4-4111-9a07-963a97b3fff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.676025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2338d6f-08a4-4111-9a07-963a97b3fff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.677794 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2338d6f-08a4-4111-9a07-963a97b3fff5 admin admin] PATH_INFO: `/v3/projects/7aa23f838ee64bfa95c120df84cd7ee9` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.686167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2338d6f-08a4-4111-9a07-963a97b3fff5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa23f838ee64bfa95c120df84cd7ee9)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.689086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2338d6f-08a4-4111-9a07-963a97b3fff5 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.833472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a2338d6f-08a4-4111-9a07-963a97b3fff5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa23f838ee64bfa95c120df84cd7ee9', 'request_id': 'req-a2338d6f-08a4-4111-9a07-963a97b3fff5'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:49.857821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a2338d6f-08a4-4111-9a07-963a97b3fff5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa23f838ee64bfa95c120df84cd7ee9', 'request_id': 'req-a2338d6f-08a4-4111-9a07-963a97b3fff5'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:49.876507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a2338d6f-08a4-4111-9a07-963a97b3fff5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa23f838ee64bfa95c120df84cd7ee9', 'request_id': 'req-a2338d6f-08a4-4111-9a07-963a97b3fff5'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:49.900919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a2338d6f-08a4-4111-9a07-963a97b3fff5 admin admin] The token cache is being invalidate because project 7aa23f838ee64bfa95c120df84cd7ee9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:49:49.901823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a2338d6f-08a4-4111-9a07-963a97b3fff5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:49.904669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a2338d6f-08a4-4111-9a07-963a97b3fff5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:49.910054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 850/1670] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:49:49 2022] DELETE /identity/v3/projects/7aa23f838ee64bfa95c120df84cd7ee9 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:49.911283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3507e7-c493-41b8-91ba-d96b1e9ae965 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:49.930175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3507e7-c493-41b8-91ba-d96b1e9ae965 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:50.101736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374c8b4-9719-4b37-894e-faf447d2f7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:50.103238 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374c8b4-9719-4b37-894e-faf447d2f7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:50.103875 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374c8b4-9719-4b37-894e-faf447d2f7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:50.114380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374c8b4-9719-4b37-894e-faf447d2f7cf admin admin] PATH_INFO: `/v3/users/f3275a034b8c4cc6858cc285d2127626` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:50.120014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374c8b4-9719-4b37-894e-faf447d2f7cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3275a034b8c4cc6858cc285d2127626)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:50.124389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374c8b4-9719-4b37-894e-faf447d2f7cf admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:50.278222 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b374c8b4-9719-4b37-894e-faf447d2f7cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3275a034b8c4cc6858cc285d2127626'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:50.289220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b374c8b4-9719-4b37-894e-faf447d2f7cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3275a034b8c4cc6858cc285d2127626'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:50.292151 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b374c8b4-9719-4b37-894e-faf447d2f7cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3275a034b8c4cc6858cc285d2127626'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:50.315256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:50.338346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b374c8b4-9719-4b37-894e-faf447d2f7cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3275a034b8c4cc6858cc285d2127626'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:50.365150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b374c8b4-9719-4b37-894e-faf447d2f7cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3275a034b8c4cc6858cc285d2127626'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:50.387086 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 821/1671] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:49:49 2022] DELETE /identity/v3/users/f3275a034b8c4cc6858cc285d2127626 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:49:51.677215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264a46c-77dc-4691-9dfe-5c4078d84f90 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.677215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264a46c-77dc-4691-9dfe-5c4078d84f90 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.684012 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264a46c-77dc-4691-9dfe-5c4078d84f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.815761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-0264a46c-77dc-4691-9dfe-5c4078d84f90 None None] MFA Rules not processed for user `a1d7a67f595b44a89abe80bc4f714821`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:51.817292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-0264a46c-77dc-4691-9dfe-5c4078d84f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:51.977254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:52.033494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 851/1672] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:52.045343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724726d3-4ddb-4879-9c2b-c57f9f010b9e None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.045929 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724726d3-4ddb-4879-9c2b-c57f9f010b9e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.046103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724726d3-4ddb-4879-9c2b-c57f9f010b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.098994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-724726d3-4ddb-4879-9c2b-c57f9f010b9e None None] MFA Rules not processed for user `a1d7a67f595b44a89abe80bc4f714821`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:52.107618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-724726d3-4ddb-4879-9c2b-c57f9f010b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:52.132210 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 822/1673] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:52.145664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264a46c-77dc-4691-9dfe-5c4078d84f90 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.148162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-0264a46c-77dc-4691-9dfe-5c4078d84f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:52.248840 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724726d3-4ddb-4879-9c2b-c57f9f010b9e None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.251585 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-724726d3-4ddb-4879-9c2b-c57f9f010b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:52.378857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264a46c-77dc-4691-9dfe-5c4078d84f90 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:52.397835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01795441-dadc-4e3a-98ac-0314933ebfc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.398452 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795441-dadc-4e3a-98ac-0314933ebfc7 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.398767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795441-dadc-4e3a-98ac-0314933ebfc7 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.399036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01795441-dadc-4e3a-98ac-0314933ebfc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.425048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01795441-dadc-4e3a-98ac-0314933ebfc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:52.432712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01795441-dadc-4e3a-98ac-0314933ebfc7 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:52.468968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 852/1674] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:49:52.489544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724726d3-4ddb-4879-9c2b-c57f9f010b9e None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:52.497967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.499067 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.499489 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.499924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] PATH_INFO: `/v3/projects/956d17ef27524da19122aba5f6b83e66` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.501970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=956d17ef27524da19122aba5f6b83e66)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:52.504297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:52.607045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '956d17ef27524da19122aba5f6b83e66', 'request_id': 'req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:52.616636 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '956d17ef27524da19122aba5f6b83e66', 'request_id': 'req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:52.629519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '956d17ef27524da19122aba5f6b83e66', 'request_id': 'req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:52.667665 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] The token cache is being invalidate because project 956d17ef27524da19122aba5f6b83e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:49:52.668353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:52.668883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:52.679278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b41a4d-074e-4a3e-9821-f3604175c836 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.680532 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b41a4d-074e-4a3e-9821-f3604175c836 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.681032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b41a4d-074e-4a3e-9821-f3604175c836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.690190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 823/1675] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:49:52 2022] DELETE /identity/v3/projects/956d17ef27524da19122aba5f6b83e66 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:52.716629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e8b41a4d-074e-4a3e-9821-f3604175c836 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:52.718326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e8b41a4d-074e-4a3e-9821-f3604175c836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:52.789105 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.791117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:52.898681 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 853/1676] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:52.907087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ed69f-ed08-4db2-9def-e0b04f349588 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.907558 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ed69f-ed08-4db2-9def-e0b04f349588 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.907964 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ed69f-ed08-4db2-9def-e0b04f349588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.976954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c72ed69f-ed08-4db2-9def-e0b04f349588 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:52.978205 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c72ed69f-ed08-4db2-9def-e0b04f349588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:53.003735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6b8be-99ac-4376-8bf5-28fe5cb2fa61 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.009415 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 854/1677] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:49:53.014306 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ed69f-ed08-4db2-9def-e0b04f349588 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.022978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f767ae-fb48-4bae-a1ea-80f0136ae0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.024026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f767ae-fb48-4bae-a1ea-80f0136ae0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.024400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f767ae-fb48-4bae-a1ea-80f0136ae0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.024824 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f767ae-fb48-4bae-a1ea-80f0136ae0b6 admin admin] PATH_INFO: `/v3/users/6ecba1d2ea2c493c969522f33f33d6de` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.031828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ed69f-ed08-4db2-9def-e0b04f349588 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.047363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f767ae-fb48-4bae-a1ea-80f0136ae0b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ecba1d2ea2c493c969522f33f33d6de)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.049404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d09b8c-1664-44ec-9a7c-d719e4aaf182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.050344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d09b8c-1664-44ec-9a7c-d719e4aaf182 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.050835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d09b8c-1664-44ec-9a7c-d719e4aaf182 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.051290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d09b8c-1664-44ec-9a7c-d719e4aaf182 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.052725 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d09b8c-1664-44ec-9a7c-d719e4aaf182 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:53.053828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f767ae-fb48-4bae-a1ea-80f0136ae0b6 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.054553 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d09b8c-1664-44ec-9a7c-d719e4aaf182 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.062138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d09b8c-1664-44ec-9a7c-d719e4aaf182 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.079946 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.136268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 855/1678] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:53.168135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d09b8c-1664-44ec-9a7c-d719e4aaf182 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.209433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d09b8c-1664-44ec-9a7c-d719e4aaf182 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.226218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da313c-d805-404a-a49d-6e4ddbafc1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.228933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da313c-d805-404a-a49d-6e4ddbafc1eb admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.229895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da313c-d805-404a-a49d-6e4ddbafc1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.229895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da313c-d805-404a-a49d-6e4ddbafc1eb admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.230659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da313c-d805-404a-a49d-6e4ddbafc1eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.233532 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da313c-d805-404a-a49d-6e4ddbafc1eb admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.255451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-00f767ae-fb48-4bae-a1ea-80f0136ae0b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ecba1d2ea2c493c969522f33f33d6de'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:53.293436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-00f767ae-fb48-4bae-a1ea-80f0136ae0b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ecba1d2ea2c493c969522f33f33d6de'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:53.299076 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-00f767ae-fb48-4bae-a1ea-80f0136ae0b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ecba1d2ea2c493c969522f33f33d6de'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:53.343634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 856/1679] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:53 2022] POST /identity/v3/projects => generated 363 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:53.345696 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da313c-d805-404a-a49d-6e4ddbafc1eb admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.347299 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-00f767ae-fb48-4bae-a1ea-80f0136ae0b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ecba1d2ea2c493c969522f33f33d6de'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:53.380092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-00f767ae-fb48-4bae-a1ea-80f0136ae0b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ecba1d2ea2c493c969522f33f33d6de'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:53.395358 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 824/1680] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:49:52 2022] DELETE /identity/v3/users/6ecba1d2ea2c493c969522f33f33d6de => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:53.401211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6952db-084e-45ed-8528-f5b39135cb26 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.402720 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6952db-084e-45ed-8528-f5b39135cb26 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.403083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6952db-084e-45ed-8528-f5b39135cb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.456454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-db6952db-084e-45ed-8528-f5b39135cb26 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:53.458645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-db6952db-084e-45ed-8528-f5b39135cb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:53.503075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.590211 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da313c-d805-404a-a49d-6e4ddbafc1eb admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.634659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 825/1681] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:53.637321 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7e5ce-1898-40b4-981b-b9fbaea704d2 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.637880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7e5ce-1898-40b4-981b-b9fbaea704d2 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.638295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7e5ce-1898-40b4-981b-b9fbaea704d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.703934 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-f6f7e5ce-1898-40b4-981b-b9fbaea704d2 None None] MFA Rules not processed for user `e8f7435928a74258af5478959bd939bb`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:53.705180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f6f7e5ce-1898-40b4-981b-b9fbaea704d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:53.756196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84608eb-f5c3-4876-a6b0-dc46a44561c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.756975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84608eb-f5c3-4876-a6b0-dc46a44561c7 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.757172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84608eb-f5c3-4876-a6b0-dc46a44561c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.757404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84608eb-f5c3-4876-a6b0-dc46a44561c7 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.758087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84608eb-f5c3-4876-a6b0-dc46a44561c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.760103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84608eb-f5c3-4876-a6b0-dc46a44561c7 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.822697 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 857/1682] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:53 2022] POST /identity/v3/users => generated 282 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:53.829592 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84608eb-f5c3-4876-a6b0-dc46a44561c7 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.833035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d84608eb-f5c3-4876-a6b0-dc46a44561c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:53.907396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 826/1683] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:53.908698 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae0b79-84ff-4447-98b4-8bf7a15607f7 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.908984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae0b79-84ff-4447-98b4-8bf7a15607f7 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.909262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae0b79-84ff-4447-98b4-8bf7a15607f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.948378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84608eb-f5c3-4876-a6b0-dc46a44561c7 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.961885 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd031bf-1c72-47bb-8bb1-253cd02f56c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.971927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd031bf-1c72-47bb-8bb1-253cd02f56c6 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.973091 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd031bf-1c72-47bb-8bb1-253cd02f56c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.973937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-79ae0b79-84ff-4447-98b4-8bf7a15607f7 None None] MFA Rules not processed for user `074f3ddb689c4a60a05d3267080d7e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:53.974047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd031bf-1c72-47bb-8bb1-253cd02f56c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.975231 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-79ae0b79-84ff-4447-98b4-8bf7a15607f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:53.990804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.997314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd031bf-1c72-47bb-8bb1-253cd02f56c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.002358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd031bf-1c72-47bb-8bb1-253cd02f56c6 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.017747 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 858/1684] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:54.021196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd031bf-1c72-47bb-8bb1-253cd02f56c6 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.024171 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ecd031bf-1c72-47bb-8bb1-253cd02f56c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:54.041765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd031bf-1c72-47bb-8bb1-253cd02f56c6 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.051369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cdb7d-6605-4b74-b7fe-4ce964a2dae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.052267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cdb7d-6605-4b74-b7fe-4ce964a2dae3 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.052683 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cdb7d-6605-4b74-b7fe-4ce964a2dae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.053068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cdb7d-6605-4b74-b7fe-4ce964a2dae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.053995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cdb7d-6605-4b74-b7fe-4ce964a2dae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.056495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cdb7d-6605-4b74-b7fe-4ce964a2dae3 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.071195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 859/1685] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:54.076023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cdb7d-6605-4b74-b7fe-4ce964a2dae3 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.078592 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-070cdb7d-6605-4b74-b7fe-4ce964a2dae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:54.122712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 827/1686] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:54.134888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae0b79-84ff-4447-98b4-8bf7a15607f7 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.152917 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae0b79-84ff-4447-98b4-8bf7a15607f7 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.166135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:54.166393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b8fa1-e092-435c-91d3-66300a275954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.166984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b8fa1-e092-435c-91d3-66300a275954 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.167292 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b8fa1-e092-435c-91d3-66300a275954 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.168125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b8fa1-e092-435c-91d3-66300a275954 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.168846 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b8fa1-e092-435c-91d3-66300a275954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.170589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b8fa1-e092-435c-91d3-66300a275954 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.206715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 828/1687] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:54.213600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b8fa1-e092-435c-91d3-66300a275954 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.214881 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cdb7d-6605-4b74-b7fe-4ce964a2dae3 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.234559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1568d-0003-4f47-bd1d-5e1ebca07e94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.239087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1568d-0003-4f47-bd1d-5e1ebca07e94 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.239442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1568d-0003-4f47-bd1d-5e1ebca07e94 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.239720 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1568d-0003-4f47-bd1d-5e1ebca07e94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.249480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b8fa1-e092-435c-91d3-66300a275954 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.264119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1568d-0003-4f47-bd1d-5e1ebca07e94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.266145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1568d-0003-4f47-bd1d-5e1ebca07e94 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.267712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76dd38-51e1-4e76-adc5-5977a6906090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.273013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76dd38-51e1-4e76-adc5-5977a6906090 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.273541 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76dd38-51e1-4e76-adc5-5977a6906090 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.278139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76dd38-51e1-4e76-adc5-5977a6906090 admin admin] PATH_INFO: `/v3/projects/824b912948864c459b5938d81bdcf653/users/9b026313923e46be9a6ab68f201727a9/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.281892 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 860/1688] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:54 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:54.339401 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76dd38-51e1-4e76-adc5-5977a6906090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=824b912948864c459b5938d81bdcf653, user_id=9b026313923e46be9a6ab68f201727a9, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.352510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76dd38-51e1-4e76-adc5-5977a6906090 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.398208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 829/1689] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:54 2022] PUT /identity/v3/projects/824b912948864c459b5938d81bdcf653/users/9b026313923e46be9a6ab68f201727a9/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:54.408251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff1fe-d7a0-446e-b3b7-c60fa836d4fc None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.408634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff1fe-d7a0-446e-b3b7-c60fa836d4fc None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.408883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff1fe-d7a0-446e-b3b7-c60fa836d4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.475684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e5eff1fe-d7a0-446e-b3b7-c60fa836d4fc None None] MFA Rules not processed for user `9b026313923e46be9a6ab68f201727a9`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:54.478538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e5eff1fe-d7a0-446e-b3b7-c60fa836d4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:54.713513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4bcb4-2a4b-41e9-aa0c-747efb561bca None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.717893 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4bcb4-2a4b-41e9-aa0c-747efb561bca None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.718267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4bcb4-2a4b-41e9-aa0c-747efb561bca None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.720667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 830/1690] 10.222.0.86 () {54 vars in 912 bytes} [Fri Aug 5 19:49:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:54.721007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 861/1691] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:49:54.726786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4bcb4-2a4b-41e9-aa0c-747efb561bca None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.734654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d883b0-04bf-48f8-bc37-18ef6dcee671 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.736432 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d883b0-04bf-48f8-bc37-18ef6dcee671 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.736740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d883b0-04bf-48f8-bc37-18ef6dcee671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.877626 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-a6d883b0-04bf-48f8-bc37-18ef6dcee671 None None] MFA Rules not processed for user `80097afc3ce540e6a8060f696535e62c`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:54.881065 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a6d883b0-04bf-48f8-bc37-18ef6dcee671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:54.978516 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4bcb4-2a4b-41e9-aa0c-747efb561bca None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.001429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8fc14-98cd-4715-b6c1-37eac3fdc198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.002288 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8fc14-98cd-4715-b6c1-37eac3fdc198 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.004198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8fc14-98cd-4715-b6c1-37eac3fdc198 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.004891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8fc14-98cd-4715-b6c1-37eac3fdc198 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.007258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8fc14-98cd-4715-b6c1-37eac3fdc198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.014104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8fc14-98cd-4715-b6c1-37eac3fdc198 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:55.045204 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 862/1692] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:49:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:55.048581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d883b0-04bf-48f8-bc37-18ef6dcee671 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.114264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 831/1693] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:49:54 2022] POST /identity/v3/projects => generated 363 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:55.115476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8fc14-98cd-4715-b6c1-37eac3fdc198 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.119180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-dde8fc14-98cd-4715-b6c1-37eac3fdc198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:55.303587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:55.315587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8fc14-98cd-4715-b6c1-37eac3fdc198 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.322462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d883b0-04bf-48f8-bc37-18ef6dcee671 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.330814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.332672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.334958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.342322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] PATH_INFO: `/v3/projects/664cf2ad8cda4ee089a12d107a785b66` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.346779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=664cf2ad8cda4ee089a12d107a785b66)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.346909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:55.349670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:55.490372 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff9104-dc98-431c-89cc-db9b87382d85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.491035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff9104-dc98-431c-89cc-db9b87382d85 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.491344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff9104-dc98-431c-89cc-db9b87382d85 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.491601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff9104-dc98-431c-89cc-db9b87382d85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.499744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '664cf2ad8cda4ee089a12d107a785b66', 'request_id': 'req-78e3f0b1-81b8-449c-8125-6f823b962414'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:55.521167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '664cf2ad8cda4ee089a12d107a785b66', 'request_id': 'req-78e3f0b1-81b8-449c-8125-6f823b962414'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:55.545107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '664cf2ad8cda4ee089a12d107a785b66', 'request_id': 'req-78e3f0b1-81b8-449c-8125-6f823b962414'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:55.572494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] The token cache is being invalidate because project 664cf2ad8cda4ee089a12d107a785b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:49:55.572870 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:55.574266 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:55.585570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 832/1694] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:49:55 2022] DELETE /identity/v3/projects/664cf2ad8cda4ee089a12d107a785b66 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:55.586885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.591790 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:55.673568 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff9104-dc98-431c-89cc-db9b87382d85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.675387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff9104-dc98-431c-89cc-db9b87382d85 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:55.689017 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 863/1695] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:55.702807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff9104-dc98-431c-89cc-db9b87382d85 service nova] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.706448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-93ff9104-dc98-431c-89cc-db9b87382d85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:55.842723 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3f0b1-81b8-449c-8125-6f823b962414 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.853103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6447547-bb69-457f-b387-6dd0ed9e4eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.853797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6447547-bb69-457f-b387-6dd0ed9e4eae admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.854081 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6447547-bb69-457f-b387-6dd0ed9e4eae admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.854351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6447547-bb69-457f-b387-6dd0ed9e4eae admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.855262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6447547-bb69-457f-b387-6dd0ed9e4eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.857500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6447547-bb69-457f-b387-6dd0ed9e4eae admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:55.918257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff9104-dc98-431c-89cc-db9b87382d85 service nova] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.919503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 833/1696] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:49:55 2022] POST /identity/v3/users => generated 288 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:55.920672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6447547-bb69-457f-b387-6dd0ed9e4eae admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.922837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c6447547-bb69-457f-b387-6dd0ed9e4eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:55.951239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6447547-bb69-457f-b387-6dd0ed9e4eae admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.967108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.967334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.967576 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.967852 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] PATH_INFO: `/v3/users/867cdfbc077940dd8947140a7416c786` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.970295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=867cdfbc077940dd8947140a7416c786)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.983441 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:56.055312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20180905-2fc6-485e-bb1c-285f8cf7e118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.056374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20180905-2fc6-485e-bb1c-285f8cf7e118 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.057470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20180905-2fc6-485e-bb1c-285f8cf7e118 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.058921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20180905-2fc6-485e-bb1c-285f8cf7e118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.063213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-20180905-2fc6-485e-bb1c-285f8cf7e118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:56.102439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20180905-2fc6-485e-bb1c-285f8cf7e118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:56.115133 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20180905-2fc6-485e-bb1c-285f8cf7e118 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:56.162370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 864/1697] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:56.165593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20180905-2fc6-485e-bb1c-285f8cf7e118 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.185277 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '867cdfbc077940dd8947140a7416c786'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:56.205026 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20180905-2fc6-485e-bb1c-285f8cf7e118 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.220863 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebb8c4-5bd5-424d-8759-cd042db135d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.222419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebb8c4-5bd5-424d-8759-cd042db135d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.223138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebb8c4-5bd5-424d-8759-cd042db135d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.223791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebb8c4-5bd5-424d-8759-cd042db135d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.226092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebb8c4-5bd5-424d-8759-cd042db135d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:56.229423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '867cdfbc077940dd8947140a7416c786'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:56.230972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '867cdfbc077940dd8947140a7416c786'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:56.231774 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebb8c4-5bd5-424d-8759-cd042db135d7 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:56.236770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:56.281041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 865/1698] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:49:56.282264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=81,keystone:DELETE=15,keystone:INSERT=4 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:56.286604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebb8c4-5bd5-424d-8759-cd042db135d7 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.288821 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '867cdfbc077940dd8947140a7416c786'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:56.312924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebb8c4-5bd5-424d-8759-cd042db135d7 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.318421 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '867cdfbc077940dd8947140a7416c786'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:56.331413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 834/1699] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:49:55 2022] DELETE /identity/v3/users/867cdfbc077940dd8947140a7416c786 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:56.332259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fb31e-004d-4dc4-8c32-3c85a5ffe908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.333098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fb31e-004d-4dc4-8c32-3c85a5ffe908 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.333489 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fb31e-004d-4dc4-8c32-3c85a5ffe908 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.333822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fb31e-004d-4dc4-8c32-3c85a5ffe908 admin admin] PATH_INFO: `/v3/projects/ad9de95bcbc44fe4bb84af374d5448e9/users/002970fb483f4626aa80f6f5448d5f05/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.376658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fb31e-004d-4dc4-8c32-3c85a5ffe908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad9de95bcbc44fe4bb84af374d5448e9, user_id=002970fb483f4626aa80f6f5448d5f05, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:56.380303 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fb31e-004d-4dc4-8c32-3c85a5ffe908 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:56.406088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 866/1700] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:56 2022] PUT /identity/v3/projects/ad9de95bcbc44fe4bb84af374d5448e9/users/002970fb483f4626aa80f6f5448d5f05/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:56.412329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.414179 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:56.611543 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74e6ca-a7a6-4d33-9235-e6b8669516cc admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.828266 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e74aa-f6c2-44df-9e20-e0db58009435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.829664 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e74aa-f6c2-44df-9e20-e0db58009435 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.830346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e74aa-f6c2-44df-9e20-e0db58009435 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.831060 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e74aa-f6c2-44df-9e20-e0db58009435 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.832595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e74aa-f6c2-44df-9e20-e0db58009435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:56.836108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e74aa-f6c2-44df-9e20-e0db58009435 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:56.890132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 835/1701] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 479 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:49:56.894001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fb31e-004d-4dc4-8c32-3c85a5ffe908 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.914890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fb31e-004d-4dc4-8c32-3c85a5ffe908 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.951437 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4024a0-c1cb-4014-8dd5-7a6de009c8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.952138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4024a0-c1cb-4014-8dd5-7a6de009c8ce admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.952422 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4024a0-c1cb-4014-8dd5-7a6de009c8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.955387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4024a0-c1cb-4014-8dd5-7a6de009c8ce admin admin] PATH_INFO: `/v3/projects/ad9de95bcbc44fe4bb84af374d5448e9/users/002970fb483f4626aa80f6f5448d5f05/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.962189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4024a0-c1cb-4014-8dd5-7a6de009c8ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad9de95bcbc44fe4bb84af374d5448e9, user_id=002970fb483f4626aa80f6f5448d5f05, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:56.980538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4024a0-c1cb-4014-8dd5-7a6de009c8ce admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.041541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 867/1702] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:49:56 2022] PUT /identity/v3/projects/ad9de95bcbc44fe4bb84af374d5448e9/users/002970fb483f4626aa80f6f5448d5f05/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:57.053191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff700b-a5ac-484f-b86f-624a94550379 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.053979 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff700b-a5ac-484f-b86f-624a94550379 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.054576 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff700b-a5ac-484f-b86f-624a94550379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.205039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-faff700b-a5ac-484f-b86f-624a94550379 None None] MFA Rules not processed for user `002970fb483f4626aa80f6f5448d5f05`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:57.213055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-faff700b-a5ac-484f-b86f-624a94550379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:57.580635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 836/1703] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 526 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:57.604355 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4024a0-c1cb-4014-8dd5-7a6de009c8ce admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.606954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7f4024a0-c1cb-4014-8dd5-7a6de009c8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:57.777200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4024a0-c1cb-4014-8dd5-7a6de009c8ce admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.787021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917f7c1-4686-4f59-b7df-adb3034c59a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.788312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917f7c1-4686-4f59-b7df-adb3034c59a1 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.788809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917f7c1-4686-4f59-b7df-adb3034c59a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.789191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917f7c1-4686-4f59-b7df-adb3034c59a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.804502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917f7c1-4686-4f59-b7df-adb3034c59a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.806577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917f7c1-4686-4f59-b7df-adb3034c59a1 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.833430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 868/1704] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:49:58.302586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c6bb2-9df9-4f30-8c2c-6e9fe8e55eab None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.303066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c6bb2-9df9-4f30-8c2c-6e9fe8e55eab None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.304240 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c6bb2-9df9-4f30-8c2c-6e9fe8e55eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.410458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-fc7c6bb2-9df9-4f30-8c2c-6e9fe8e55eab None None] MFA Rules not processed for user `a357a3d27e4e41adae1c56ee39ae5aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:58.412156 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-fc7c6bb2-9df9-4f30-8c2c-6e9fe8e55eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:58.417286 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.455266 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917f7c1-4686-4f59-b7df-adb3034c59a1 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.461236 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c917f7c1-4686-4f59-b7df-adb3034c59a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:58.599699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 837/1705] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:58.604779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917f7c1-4686-4f59-b7df-adb3034c59a1 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.606947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c6bb2-9df9-4f30-8c2c-6e9fe8e55eab None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.608610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-fc7c6bb2-9df9-4f30-8c2c-6e9fe8e55eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:58.613973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cfb310-a6c2-4352-8349-c0b2d23bccff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.614976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfb310-a6c2-4352-8349-c0b2d23bccff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.615336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfb310-a6c2-4352-8349-c0b2d23bccff admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.615678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cfb310-a6c2-4352-8349-c0b2d23bccff admin admin] PATH_INFO: `/v3/projects/507ce3e3dce6428bbf81c1f4b550a5fc` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.617551 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfb310-a6c2-4352-8349-c0b2d23bccff admin admin] RBAC: Authorizing `identity:delete_project(project_id=507ce3e3dce6428bbf81c1f4b550a5fc)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.621178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cfb310-a6c2-4352-8349-c0b2d23bccff admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.720262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a6cfb310-a6c2-4352-8349-c0b2d23bccff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '507ce3e3dce6428bbf81c1f4b550a5fc', 'request_id': 'req-a6cfb310-a6c2-4352-8349-c0b2d23bccff'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:58.728242 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a6cfb310-a6c2-4352-8349-c0b2d23bccff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '507ce3e3dce6428bbf81c1f4b550a5fc', 'request_id': 'req-a6cfb310-a6c2-4352-8349-c0b2d23bccff'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:58.763740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a6cfb310-a6c2-4352-8349-c0b2d23bccff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '507ce3e3dce6428bbf81c1f4b550a5fc', 'request_id': 'req-a6cfb310-a6c2-4352-8349-c0b2d23bccff'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:58.768248 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.827197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a6cfb310-a6c2-4352-8349-c0b2d23bccff admin admin] The token cache is being invalidate because project 507ce3e3dce6428bbf81c1f4b550a5fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:49:58.827522 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a6cfb310-a6c2-4352-8349-c0b2d23bccff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:58.831750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a6cfb310-a6c2-4352-8349-c0b2d23bccff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:58.837237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c6bb2-9df9-4f30-8c2c-6e9fe8e55eab None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.838164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 869/1706] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:49:58 2022] DELETE /identity/v3/projects/507ce3e3dce6428bbf81c1f4b550a5fc => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:49:58.843945 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ccc20-185f-415d-b28e-88c3fb54c9a5 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.844293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ccc20-185f-415d-b28e-88c3fb54c9a5 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.844492 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ccc20-185f-415d-b28e-88c3fb54c9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.908995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3b9ccc20-185f-415d-b28e-88c3fb54c9a5 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:58.910230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3b9ccc20-185f-415d-b28e-88c3fb54c9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:59.141568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a83b2-745d-4fd2-82b6-1b9fe843689f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:59.142216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a83b2-745d-4fd2-82b6-1b9fe843689f service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.142498 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a83b2-745d-4fd2-82b6-1b9fe843689f service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.142917 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a83b2-745d-4fd2-82b6-1b9fe843689f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.153329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9f6a83b2-745d-4fd2-82b6-1b9fe843689f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:59.157038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 870/1707] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:59.160811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ccc20-185f-415d-b28e-88c3fb54c9a5 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:59.168202 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3b9ccc20-185f-415d-b28e-88c3fb54c9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:59.203817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ccc20-185f-415d-b28e-88c3fb54c9a5 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:59.213339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226bac8-28f3-48a2-866d-22c0460332ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:59.215582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226bac8-28f3-48a2-866d-22c0460332ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.216071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226bac8-28f3-48a2-866d-22c0460332ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.216427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226bac8-28f3-48a2-866d-22c0460332ae admin admin] PATH_INFO: `/v3/users/9b026313923e46be9a6ab68f201727a9` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.217733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226bac8-28f3-48a2-866d-22c0460332ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b026313923e46be9a6ab68f201727a9)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:59.220242 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226bac8-28f3-48a2-866d-22c0460332ae admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:59.288524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:59.324851 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2226bac8-28f3-48a2-866d-22c0460332ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b026313923e46be9a6ab68f201727a9'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:59.351432 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2226bac8-28f3-48a2-866d-22c0460332ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b026313923e46be9a6ab68f201727a9'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:59.358984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a83b2-745d-4fd2-82b6-1b9fe843689f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:59.360614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a83b2-745d-4fd2-82b6-1b9fe843689f service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:59.372874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2226bac8-28f3-48a2-866d-22c0460332ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b026313923e46be9a6ab68f201727a9'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:59.374535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2226bac8-28f3-48a2-866d-22c0460332ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b026313923e46be9a6ab68f201727a9'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:59.375593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2226bac8-28f3-48a2-866d-22c0460332ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b026313923e46be9a6ab68f201727a9'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:49:59.399033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 838/1708] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 792 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:49:59.400541 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf1f7e-5ef5-4b6f-b1f6-5542ce9b9fa8 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.400827 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf1f7e-5ef5-4b6f-b1f6-5542ce9b9fa8 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.401112 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf1f7e-5ef5-4b6f-b1f6-5542ce9b9fa8 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.408300 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 871/1709] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:49:59 2022] DELETE /identity/v3/users/9b026313923e46be9a6ab68f201727a9 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:59.409831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee493-cf7d-47b3-ab99-4d79eb12e396 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.410116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee493-cf7d-47b3-ab99-4d79eb12e396 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.410383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee493-cf7d-47b3-ab99-4d79eb12e396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.411422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 839/1710] 10.222.0.86 () {54 vars in 961 bytes} [Fri Aug 5 19:49:59 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:59.412813 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593c21a-cb44-4e78-b56c-78b14e4624ab None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.413102 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593c21a-cb44-4e78-b56c-78b14e4624ab None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.413373 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593c21a-cb44-4e78-b56c-78b14e4624ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.475967 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-52fee493-cf7d-47b3-ab99-4d79eb12e396 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:59.479381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-52fee493-cf7d-47b3-ab99-4d79eb12e396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:59.480806 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-e593c21a-cb44-4e78-b56c-78b14e4624ab None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:59.492122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e593c21a-cb44-4e78-b56c-78b14e4624ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:59.649841 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 840/1711] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:59.652056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de27e68-7881-4a6e-86ff-e1240e0f0365 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.653358 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de27e68-7881-4a6e-86ff-e1240e0f0365 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.654159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de27e68-7881-4a6e-86ff-e1240e0f0365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.716374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-3de27e68-7881-4a6e-86ff-e1240e0f0365 None None] MFA Rules not processed for user `cce71f93c80d433b830c448d0a7a14e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:59.722921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3de27e68-7881-4a6e-86ff-e1240e0f0365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:59.758362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 872/1712] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:59.764641 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fee493-cf7d-47b3-ab99-4d79eb12e396 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:59.768133 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-52fee493-cf7d-47b3-ab99-4d79eb12e396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:59.785383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:59.913694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 841/1713] 10.222.0.86 () {58 vars in 1058 bytes} [Fri Aug 5 19:49:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:49:59.921421 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de27e68-7881-4a6e-86ff-e1240e0f0365 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:59.956175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de27e68-7881-4a6e-86ff-e1240e0f0365 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:59.967110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72232c9-dfb6-45a2-98f9-a5bab28ddd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:59.968882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72232c9-dfb6-45a2-98f9-a5bab28ddd45 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:59.970965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72232c9-dfb6-45a2-98f9-a5bab28ddd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:59.972044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72232c9-dfb6-45a2-98f9-a5bab28ddd45 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:59.976456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72232c9-dfb6-45a2-98f9-a5bab28ddd45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:59.985034 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72232c9-dfb6-45a2-98f9-a5bab28ddd45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:59.987383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72232c9-dfb6-45a2-98f9-a5bab28ddd45 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:59.990337 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fee493-cf7d-47b3-ab99-4d79eb12e396 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:59.997580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350aa46-b687-431a-9958-e639f4ce63c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:59.998930 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350aa46-b687-431a-9958-e639f4ce63c4 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.001692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350aa46-b687-431a-9958-e639f4ce63c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.002812 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350aa46-b687-431a-9958-e639f4ce63c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.010575 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 842/1714] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:00.011690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72232c9-dfb6-45a2-98f9-a5bab28ddd45 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.039257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350aa46-b687-431a-9958-e639f4ce63c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.042305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72232c9-dfb6-45a2-98f9-a5bab28ddd45 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.044235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350aa46-b687-431a-9958-e639f4ce63c4 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.062416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89ec78-a537-420c-ad13-423483cb4283 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.063231 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89ec78-a537-420c-ad13-423483cb4283 service placement] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.063610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89ec78-a537-420c-ad13-423483cb4283 service placement] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.063985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89ec78-a537-420c-ad13-423483cb4283 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.067284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ab89ec78-a537-420c-ad13-423483cb4283 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:00.069870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 873/1715] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:00.082822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350aa46-b687-431a-9958-e639f4ce63c4 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.097310 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350aa46-b687-431a-9958-e639f4ce63c4 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.107697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89ec78-a537-420c-ad13-423483cb4283 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.108495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff60595-d0bc-4cf8-8815-d111b49afdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.109326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff60595-d0bc-4cf8-8815-d111b49afdef admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.109612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff60595-d0bc-4cf8-8815-d111b49afdef admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.109888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff60595-d0bc-4cf8-8815-d111b49afdef admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.110695 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff60595-d0bc-4cf8-8815-d111b49afdef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.112690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89ec78-a537-420c-ad13-423483cb4283 service placement] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.113102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff60595-d0bc-4cf8-8815-d111b49afdef admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.173755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 843/1716] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:00.176008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 874/1717] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:00 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:00.180282 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89ec78-a537-420c-ad13-423483cb4283 service placement] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.212376 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89ec78-a537-420c-ad13-423483cb4283 service placement] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.241582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f3f4f-326f-4159-9d5f-159b7a008175 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.242476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f3f4f-326f-4159-9d5f-159b7a008175 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.242796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f3f4f-326f-4159-9d5f-159b7a008175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.316703 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-fe4f3f4f-326f-4159-9d5f-159b7a008175 None None] MFA Rules not processed for user `d183fcdae86448caa81c09823dd3884e`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:00.319126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fe4f3f4f-326f-4159-9d5f-159b7a008175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:00.327459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:00.347184 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab755ed2-b61b-4328-a640-885337b5a1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.347833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab755ed2-b61b-4328-a640-885337b5a1f8 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.348114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab755ed2-b61b-4328-a640-885337b5a1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.348384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab755ed2-b61b-4328-a640-885337b5a1f8 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.349066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab755ed2-b61b-4328-a640-885337b5a1f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.351061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab755ed2-b61b-4328-a640-885337b5a1f8 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.405553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 844/1718] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:00 2022] POST /identity/v3/users => generated 283 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:00.410479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab755ed2-b61b-4328-a640-885337b5a1f8 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.442003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab755ed2-b61b-4328-a640-885337b5a1f8 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.459502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f6b21-0f77-4b8e-88a6-3e003b9104d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.460485 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f6b21-0f77-4b8e-88a6-3e003b9104d3 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.460943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f6b21-0f77-4b8e-88a6-3e003b9104d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.461218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f6b21-0f77-4b8e-88a6-3e003b9104d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.461935 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f6b21-0f77-4b8e-88a6-3e003b9104d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.465248 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f6b21-0f77-4b8e-88a6-3e003b9104d3 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.496405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 845/1719] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:50:00.500146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f6b21-0f77-4b8e-88a6-3e003b9104d3 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.521317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f6b21-0f77-4b8e-88a6-3e003b9104d3 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.531640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 875/1720] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:00.534047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b4771-da8a-4fdf-a5e3-a2c65ff7fdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.534685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b4771-da8a-4fdf-a5e3-a2c65ff7fdf1 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.534967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b4771-da8a-4fdf-a5e3-a2c65ff7fdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.535304 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b4771-da8a-4fdf-a5e3-a2c65ff7fdf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.536019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b4771-da8a-4fdf-a5e3-a2c65ff7fdf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.537763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b4771-da8a-4fdf-a5e3-a2c65ff7fdf1 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.542120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd28aa3-96bb-43bb-a274-898b9a42c417 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.542507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd28aa3-96bb-43bb-a274-898b9a42c417 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.542725 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd28aa3-96bb-43bb-a274-898b9a42c417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.566366 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 846/1721] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:50:00.575965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b4771-da8a-4fdf-a5e3-a2c65ff7fdf1 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.592628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-ccd28aa3-96bb-43bb-a274-898b9a42c417 None None] MFA Rules not processed for user `d183fcdae86448caa81c09823dd3884e`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:00.594044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ccd28aa3-96bb-43bb-a274-898b9a42c417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:00.604787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b4771-da8a-4fdf-a5e3-a2c65ff7fdf1 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.614331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 876/1722] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:00.626113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd28aa3-96bb-43bb-a274-898b9a42c417 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.627871 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ccd28aa3-96bb-43bb-a274-898b9a42c417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:00.630176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54251822-6a85-4168-bce1-a0cab68331aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.630957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54251822-6a85-4168-bce1-a0cab68331aa admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.631238 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54251822-6a85-4168-bce1-a0cab68331aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.631505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54251822-6a85-4168-bce1-a0cab68331aa admin admin] PATH_INFO: `/v3/projects/7558ffaf30d44c37b8d2c4a7807d88f9/users/ffbb0c59812e412ba17a15a28104aa63/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.665901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54251822-6a85-4168-bce1-a0cab68331aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7558ffaf30d44c37b8d2c4a7807d88f9, user_id=ffbb0c59812e412ba17a15a28104aa63, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.668708 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54251822-6a85-4168-bce1-a0cab68331aa admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.693694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 847/1723] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:00 2022] PUT /identity/v3/projects/7558ffaf30d44c37b8d2c4a7807d88f9/users/ffbb0c59812e412ba17a15a28104aa63/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:00.705337 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f9a13-4e72-4f5b-b494-dc792fc05090 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.705591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f9a13-4e72-4f5b-b494-dc792fc05090 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.706048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f9a13-4e72-4f5b-b494-dc792fc05090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.762931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-264f9a13-4e72-4f5b-b494-dc792fc05090 None None] MFA Rules not processed for user `ffbb0c59812e412ba17a15a28104aa63`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:00.764180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-264f9a13-4e72-4f5b-b494-dc792fc05090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:00.780774 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd28aa3-96bb-43bb-a274-898b9a42c417 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.794236 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:00.937074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 848/1724] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:00.947760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f9a13-4e72-4f5b-b494-dc792fc05090 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.054842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b98f6d-1570-4ba9-bd4d-3b890864a31a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.058950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b98f6d-1570-4ba9-bd4d-3b890864a31a service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.062267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b98f6d-1570-4ba9-bd4d-3b890864a31a service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.062932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b98f6d-1570-4ba9-bd4d-3b890864a31a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.085749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f9a13-4e72-4f5b-b494-dc792fc05090 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.105363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff0e7c-f2f6-40fe-afbf-e0837f473416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.105888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff0e7c-f2f6-40fe-afbf-e0837f473416 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.106173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff0e7c-f2f6-40fe-afbf-e0837f473416 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.106564 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff0e7c-f2f6-40fe-afbf-e0837f473416 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.107299 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff0e7c-f2f6-40fe-afbf-e0837f473416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.109343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff0e7c-f2f6-40fe-afbf-e0837f473416 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.154772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 849/1725] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:00 2022] POST /identity/v3/projects => generated 363 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:01.158989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff0e7c-f2f6-40fe-afbf-e0837f473416 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.173248 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.265068 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff0e7c-f2f6-40fe-afbf-e0837f473416 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.284434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ed60a-100a-4281-99cb-23ba215164fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.294815 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ed60a-100a-4281-99cb-23ba215164fb admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.296592 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ed60a-100a-4281-99cb-23ba215164fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.297112 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ed60a-100a-4281-99cb-23ba215164fb admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.298085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ed60a-100a-4281-99cb-23ba215164fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.312288 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ed60a-100a-4281-99cb-23ba215164fb admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.337739 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b98f6d-1570-4ba9-bd4d-3b890864a31a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.340189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b98f6d-1570-4ba9-bd4d-3b890864a31a service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.375126 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 850/1726] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:01 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:01.376404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ed60a-100a-4281-99cb-23ba215164fb admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.387871 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 877/1727] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:00 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 762 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:50:01.404617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ed60a-100a-4281-99cb-23ba215164fb admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.413454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec2c2f-5045-4901-a886-f203a1c2d20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.415777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec2c2f-5045-4901-a886-f203a1c2d20f admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.416320 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec2c2f-5045-4901-a886-f203a1c2d20f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.416791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec2c2f-5045-4901-a886-f203a1c2d20f admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.418223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec2c2f-5045-4901-a886-f203a1c2d20f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.420566 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec2c2f-5045-4901-a886-f203a1c2d20f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.459794 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b98f6d-1570-4ba9-bd4d-3b890864a31a service nova] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.465220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 851/1728] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:01.483037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b98f6d-1570-4ba9-bd4d-3b890864a31a service nova] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.513257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755a568-d41a-4b4c-894a-89109642291d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.514078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755a568-d41a-4b4c-894a-89109642291d admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.514380 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755a568-d41a-4b4c-894a-89109642291d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.514677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755a568-d41a-4b4c-894a-89109642291d admin admin] PATH_INFO: `/v3/projects/54b3d9e6545e4efa889d567e0a40f173/users/b2e73b3d830b437388c236835112d007/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.570558 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755a568-d41a-4b4c-894a-89109642291d admin admin] RBAC: Authorizing `identity:create_grant(project_id=54b3d9e6545e4efa889d567e0a40f173, user_id=b2e73b3d830b437388c236835112d007, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.576163 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755a568-d41a-4b4c-894a-89109642291d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.624450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 878/1729] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:01 2022] PUT /identity/v3/projects/54b3d9e6545e4efa889d567e0a40f173/users/b2e73b3d830b437388c236835112d007/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:01.627445 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec2c2f-5045-4901-a886-f203a1c2d20f admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.645872 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec2c2f-5045-4901-a886-f203a1c2d20f admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.782533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777611c-c475-429d-881a-eb1f5023488c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.783306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777611c-c475-429d-881a-eb1f5023488c admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.783478 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777611c-c475-429d-881a-eb1f5023488c admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.783938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777611c-c475-429d-881a-eb1f5023488c admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.784701 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777611c-c475-429d-881a-eb1f5023488c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.786427 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777611c-c475-429d-881a-eb1f5023488c admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.831223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 852/1730] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:01.832226 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755a568-d41a-4b4c-894a-89109642291d admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.854101 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755a568-d41a-4b4c-894a-89109642291d admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.876771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f712b77-2030-4a9e-99a2-157f66e17d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.877263 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f712b77-2030-4a9e-99a2-157f66e17d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.877552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f712b77-2030-4a9e-99a2-157f66e17d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.880123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f712b77-2030-4a9e-99a2-157f66e17d5f admin admin] PATH_INFO: `/v3/projects/54b3d9e6545e4efa889d567e0a40f173/users/b2e73b3d830b437388c236835112d007/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.888250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f712b77-2030-4a9e-99a2-157f66e17d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=54b3d9e6545e4efa889d567e0a40f173, user_id=b2e73b3d830b437388c236835112d007, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.897941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f712b77-2030-4a9e-99a2-157f66e17d5f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.935053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8966f6c-37db-49da-84a9-a850825ba68d None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.935781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8966f6c-37db-49da-84a9-a850825ba68d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.936220 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 879/1731] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:01 2022] PUT /identity/v3/projects/54b3d9e6545e4efa889d567e0a40f173/users/b2e73b3d830b437388c236835112d007/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:01.945434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8966f6c-37db-49da-84a9-a850825ba68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.946400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6c223-6323-4d57-ab1d-cc6ea0feb94a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.946729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6c223-6323-4d57-ab1d-cc6ea0feb94a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.947186 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6c223-6323-4d57-ab1d-cc6ea0feb94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.996297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-a8966f6c-37db-49da-84a9-a850825ba68d None None] MFA Rules not processed for user `e8f7435928a74258af5478959bd939bb`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:01.997833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a8966f6c-37db-49da-84a9-a850825ba68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:02.054174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-50f6c223-6323-4d57-ab1d-cc6ea0feb94a None None] MFA Rules not processed for user `b2e73b3d830b437388c236835112d007`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:02.059932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-50f6c223-6323-4d57-ab1d-cc6ea0feb94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:02.157429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 853/1732] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:02.211521 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.260361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 880/1733] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:02.279456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8966f6c-37db-49da-84a9-a850825ba68d None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.280880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a8966f6c-37db-49da-84a9-a850825ba68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:02.408697 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6c223-6323-4d57-ab1d-cc6ea0feb94a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.410183 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.411077 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-50f6c223-6323-4d57-ab1d-cc6ea0feb94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:02.503049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8966f6c-37db-49da-84a9-a850825ba68d None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.520036 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad79380-39b1-4a5e-bbd2-f816d71e5abb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.520678 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad79380-39b1-4a5e-bbd2-f816d71e5abb service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.520995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad79380-39b1-4a5e-bbd2-f816d71e5abb service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.521267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad79380-39b1-4a5e-bbd2-f816d71e5abb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.546394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad79380-39b1-4a5e-bbd2-f816d71e5abb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.554251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad79380-39b1-4a5e-bbd2-f816d71e5abb service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.586807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 854/1734] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:02.701590 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6c223-6323-4d57-ab1d-cc6ea0feb94a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.722987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc503c1b-33ad-4fa4-a152-5731c71d7984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.727369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc503c1b-33ad-4fa4-a152-5731c71d7984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.727786 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc503c1b-33ad-4fa4-a152-5731c71d7984 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.728949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc503c1b-33ad-4fa4-a152-5731c71d7984 admin admin] PATH_INFO: `/v3/projects/824b912948864c459b5938d81bdcf653` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.734521 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc503c1b-33ad-4fa4-a152-5731c71d7984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=824b912948864c459b5938d81bdcf653)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.746754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc503c1b-33ad-4fa4-a152-5731c71d7984 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.824197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad79380-39b1-4a5e-bbd2-f816d71e5abb service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.864081 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad79380-39b1-4a5e-bbd2-f816d71e5abb service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.908906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da741d0-92b2-4827-9775-c69b51e554a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.913254 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da741d0-92b2-4827-9775-c69b51e554a3 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.913803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da741d0-92b2-4827-9775-c69b51e554a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.916769 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da741d0-92b2-4827-9775-c69b51e554a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.922193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-dc503c1b-33ad-4fa4-a152-5731c71d7984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '824b912948864c459b5938d81bdcf653', 'request_id': 'req-dc503c1b-33ad-4fa4-a152-5731c71d7984'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:02.934857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-dc503c1b-33ad-4fa4-a152-5731c71d7984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '824b912948864c459b5938d81bdcf653', 'request_id': 'req-dc503c1b-33ad-4fa4-a152-5731c71d7984'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:02.952140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-dc503c1b-33ad-4fa4-a152-5731c71d7984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '824b912948864c459b5938d81bdcf653', 'request_id': 'req-dc503c1b-33ad-4fa4-a152-5731c71d7984'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:03.017125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-dc503c1b-33ad-4fa4-a152-5731c71d7984 admin admin] The token cache is being invalidate because project 824b912948864c459b5938d81bdcf653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:03.017418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-dc503c1b-33ad-4fa4-a152-5731c71d7984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:03.019165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-dc503c1b-33ad-4fa4-a152-5731c71d7984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:03.024811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 881/1735] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:02 2022] DELETE /identity/v3/projects/824b912948864c459b5938d81bdcf653 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:03.026066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2661df-447e-474c-bda4-ff5d8a7af4ad None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.026902 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2661df-447e-474c-bda4-ff5d8a7af4ad None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.027369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2661df-447e-474c-bda4-ff5d8a7af4ad None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.043019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 882/1736] 10.222.0.86 () {54 vars in 912 bytes} [Fri Aug 5 19:50:03 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:03.043999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2661df-447e-474c-bda4-ff5d8a7af4ad None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.046127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9b2661df-447e-474c-bda4-ff5d8a7af4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:03.084473 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da741d0-92b2-4827-9775-c69b51e554a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.086118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da741d0-92b2-4827-9775-c69b51e554a3 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.104938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 855/1737] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:03.106752 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c064bb-552b-472d-bdee-60ea6b19e16b None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.107155 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c064bb-552b-472d-bdee-60ea6b19e16b None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.107502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c064bb-552b-472d-bdee-60ea6b19e16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.151121 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-31c064bb-552b-472d-bdee-60ea6b19e16b None None] MFA Rules not processed for user `80097afc3ce540e6a8060f696535e62c`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:03.159130 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-31c064bb-552b-472d-bdee-60ea6b19e16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:03.289169 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2661df-447e-474c-bda4-ff5d8a7af4ad None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.303029 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e1df1-9f48-4a6d-a0e1-1852cd8d0476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.303490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e1df1-9f48-4a6d-a0e1-1852cd8d0476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.303767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e1df1-9f48-4a6d-a0e1-1852cd8d0476 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.304039 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e1df1-9f48-4a6d-a0e1-1852cd8d0476 admin admin] PATH_INFO: `/v3/users/002970fb483f4626aa80f6f5448d5f05` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.306094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e1df1-9f48-4a6d-a0e1-1852cd8d0476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=002970fb483f4626aa80f6f5448d5f05)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.309225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e1df1-9f48-4a6d-a0e1-1852cd8d0476 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.334115 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 856/1738] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:50:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:03.369890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:03.428910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-096e1df1-9f48-4a6d-a0e1-1852cd8d0476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002970fb483f4626aa80f6f5448d5f05'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:03.467593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-096e1df1-9f48-4a6d-a0e1-1852cd8d0476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002970fb483f4626aa80f6f5448d5f05'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:03.487399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-096e1df1-9f48-4a6d-a0e1-1852cd8d0476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002970fb483f4626aa80f6f5448d5f05'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:03.488116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-096e1df1-9f48-4a6d-a0e1-1852cd8d0476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002970fb483f4626aa80f6f5448d5f05'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:03.489510 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-096e1df1-9f48-4a6d-a0e1-1852cd8d0476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002970fb483f4626aa80f6f5448d5f05'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:03.550333 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 883/1739] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:03 2022] DELETE /identity/v3/users/002970fb483f4626aa80f6f5448d5f05 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:03.911750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c064bb-552b-472d-bdee-60ea6b19e16b None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.914211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-31c064bb-552b-472d-bdee-60ea6b19e16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:03.936763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c064bb-552b-472d-bdee-60ea6b19e16b None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.047462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a107f-120e-45c4-a2dd-78605fac1d6b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.048037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a107f-120e-45c4-a2dd-78605fac1d6b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.048494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a107f-120e-45c4-a2dd-78605fac1d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.080280 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27361396-279a-4c34-b239-2d3298116fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.082101 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27361396-279a-4c34-b239-2d3298116fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.082583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27361396-279a-4c34-b239-2d3298116fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.085022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27361396-279a-4c34-b239-2d3298116fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.091395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-27361396-279a-4c34-b239-2d3298116fc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:04.117066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27361396-279a-4c34-b239-2d3298116fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.123055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27361396-279a-4c34-b239-2d3298116fc6 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.162343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 857/1740] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:04.200626 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-995a107f-120e-45c4-a2dd-78605fac1d6b None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:04.201906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-995a107f-120e-45c4-a2dd-78605fac1d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:04.354312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 884/1741] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:04.602728 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27361396-279a-4c34-b239-2d3298116fc6 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.604972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-27361396-279a-4c34-b239-2d3298116fc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:04.679247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:04.845160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27361396-279a-4c34-b239-2d3298116fc6 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.853547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf19eaf-fa92-450d-82b2-b3ec061a0c4b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.854241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf19eaf-fa92-450d-82b2-b3ec061a0c4b service swift] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.860556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf19eaf-fa92-450d-82b2-b3ec061a0c4b service swift] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.861058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf19eaf-fa92-450d-82b2-b3ec061a0c4b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.877717 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf19eaf-fa92-450d-82b2-b3ec061a0c4b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.879604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf19eaf-fa92-450d-82b2-b3ec061a0c4b service swift] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.918994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 858/1742] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:50:06.171491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a107f-120e-45c4-a2dd-78605fac1d6b None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:06.172898 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-995a107f-120e-45c4-a2dd-78605fac1d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:06.349563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a107f-120e-45c4-a2dd-78605fac1d6b None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:06.357891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:06.358573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.359428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.359721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb admin admin] PATH_INFO: `/v3/projects/ad9de95bcbc44fe4bb84af374d5448e9` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.370903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad9de95bcbc44fe4bb84af374d5448e9)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:06.374798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:06.449739 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:06.716967 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad9de95bcbc44fe4bb84af374d5448e9', 'request_id': 'req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:06.760204 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad9de95bcbc44fe4bb84af374d5448e9', 'request_id': 'req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:06.784338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad9de95bcbc44fe4bb84af374d5448e9', 'request_id': 'req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:06.825933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb admin admin] The token cache is being invalidate because project ad9de95bcbc44fe4bb84af374d5448e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:06.826364 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:06.829602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-8a0ecadb-6ecd-4fbd-b583-adb493ecb0fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:06.833278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 885/1743] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:06 2022] DELETE /identity/v3/projects/ad9de95bcbc44fe4bb84af374d5448e9 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:06.843097 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed14a34-9974-4311-8d42-7ce089e1e5cb None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.843651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed14a34-9974-4311-8d42-7ce089e1e5cb None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.843969 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed14a34-9974-4311-8d42-7ce089e1e5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.905556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-3ed14a34-9974-4311-8d42-7ce089e1e5cb None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:06.908776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3ed14a34-9974-4311-8d42-7ce089e1e5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:07.030585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb2cd0-6cb8-48e7-8a24-bba2572de005 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.031838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb2cd0-6cb8-48e7-8a24-bba2572de005 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.032418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb2cd0-6cb8-48e7-8a24-bba2572de005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.125109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3cbb2cd0-6cb8-48e7-8a24-bba2572de005 None None] MFA Rules not processed for user `d183fcdae86448caa81c09823dd3884e`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:07.126373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3cbb2cd0-6cb8-48e7-8a24-bba2572de005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:07.167736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 859/1744] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:07.174978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f6848-442a-46dd-9981-2b6a3587c813 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.178794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f6848-442a-46dd-9981-2b6a3587c813 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.178794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f6848-442a-46dd-9981-2b6a3587c813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.268613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-7c8f6848-442a-46dd-9981-2b6a3587c813 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:07.274261 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7c8f6848-442a-46dd-9981-2b6a3587c813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:07.302233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 860/1745] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:07.308111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f6848-442a-46dd-9981-2b6a3587c813 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.353097 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f6848-442a-46dd-9981-2b6a3587c813 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.380634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 886/1746] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:07.383427 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e710f-27c3-45b4-929c-cd42cac39071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.384128 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e710f-27c3-45b4-929c-cd42cac39071 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.384409 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e710f-27c3-45b4-929c-cd42cac39071 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.384684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e710f-27c3-45b4-929c-cd42cac39071 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.385856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e710f-27c3-45b4-929c-cd42cac39071 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:07.386259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e710f-27c3-45b4-929c-cd42cac39071 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.388380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e710f-27c3-45b4-929c-cd42cac39071 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.416240 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e369157-aa9d-4984-bf97-465eb4aafad6 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.416701 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e369157-aa9d-4984-bf97-465eb4aafad6 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.417717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e369157-aa9d-4984-bf97-465eb4aafad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.450937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 861/1747] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:07.451969 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e710f-27c3-45b4-929c-cd42cac39071 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.502800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-7e369157-aa9d-4984-bf97-465eb4aafad6 None None] MFA Rules not processed for user `a357a3d27e4e41adae1c56ee39ae5aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:07.504348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7e369157-aa9d-4984-bf97-465eb4aafad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:07.508300 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e710f-27c3-45b4-929c-cd42cac39071 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.527725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30ecc4-1b63-4b1b-9bf6-11eca17b6514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.528489 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30ecc4-1b63-4b1b-9bf6-11eca17b6514 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.528767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30ecc4-1b63-4b1b-9bf6-11eca17b6514 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.529033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30ecc4-1b63-4b1b-9bf6-11eca17b6514 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.534116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30ecc4-1b63-4b1b-9bf6-11eca17b6514 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.537822 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30ecc4-1b63-4b1b-9bf6-11eca17b6514 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.638120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 862/1748] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:07 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:07.638709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac376bf-49bc-4321-95fe-1bf2caf4ee6e None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.638990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac376bf-49bc-4321-95fe-1bf2caf4ee6e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.639267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac376bf-49bc-4321-95fe-1bf2caf4ee6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.681071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-3ac376bf-49bc-4321-95fe-1bf2caf4ee6e None None] MFA Rules not processed for user `074f3ddb689c4a60a05d3267080d7e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:07.684556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 887/1749] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:07.685639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e369157-aa9d-4984-bf97-465eb4aafad6 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.687442 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3ac376bf-49bc-4321-95fe-1bf2caf4ee6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:07.718882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e369157-aa9d-4984-bf97-465eb4aafad6 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.856447 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.894274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 863/1750] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:07.897985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.937493 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031c8bc-28c1-44fc-9bd0-12a4e4484c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.940732 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031c8bc-28c1-44fc-9bd0-12a4e4484c5c admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.941541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031c8bc-28c1-44fc-9bd0-12a4e4484c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.943550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031c8bc-28c1-44fc-9bd0-12a4e4484c5c admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.946781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031c8bc-28c1-44fc-9bd0-12a4e4484c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.956015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031c8bc-28c1-44fc-9bd0-12a4e4484c5c admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.030379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 888/1751] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:07 2022] POST /identity/v3/users => generated 292 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:08.032673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac376bf-49bc-4321-95fe-1bf2caf4ee6e None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.048079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac376bf-49bc-4321-95fe-1bf2caf4ee6e None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.058164 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ca3ed-7ee0-4183-a447-80421fdf12bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.059400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ca3ed-7ee0-4183-a447-80421fdf12bc admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.059701 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ca3ed-7ee0-4183-a447-80421fdf12bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.060169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ca3ed-7ee0-4183-a447-80421fdf12bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.064434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ca3ed-7ee0-4183-a447-80421fdf12bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.068656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ca3ed-7ee0-4183-a447-80421fdf12bc admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.107768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 864/1752] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:08.111715 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031c8bc-28c1-44fc-9bd0-12a4e4484c5c admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.128099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031c8bc-28c1-44fc-9bd0-12a4e4484c5c admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.141806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4143d-72f2-48c7-aee3-a4e7b6d1c46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.143606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4143d-72f2-48c7-aee3-a4e7b6d1c46f admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.143891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4143d-72f2-48c7-aee3-a4e7b6d1c46f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.144162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4143d-72f2-48c7-aee3-a4e7b6d1c46f admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.144874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4143d-72f2-48c7-aee3-a4e7b6d1c46f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.146500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4143d-72f2-48c7-aee3-a4e7b6d1c46f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.175120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ca3ed-7ee0-4183-a447-80421fdf12bc admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.187451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ca3ed-7ee0-4183-a447-80421fdf12bc admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.187622 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 889/1753] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:08.203724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a249e3-972b-4987-b809-82adcc54d9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.204807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a249e3-972b-4987-b809-82adcc54d9c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.205041 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a249e3-972b-4987-b809-82adcc54d9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.205396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a249e3-972b-4987-b809-82adcc54d9c7 admin admin] PATH_INFO: `/v3/projects/02f03bc0fa114c54afc2a696fa8bd653/users/ec846d9fc63b4e9781949f70c7bf13fa/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.241260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a249e3-972b-4987-b809-82adcc54d9c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f03bc0fa114c54afc2a696fa8bd653, user_id=ec846d9fc63b4e9781949f70c7bf13fa, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.263902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a249e3-972b-4987-b809-82adcc54d9c7 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.306853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 865/1754] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:08 2022] PUT /identity/v3/projects/02f03bc0fa114c54afc2a696fa8bd653/users/ec846d9fc63b4e9781949f70c7bf13fa/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:08.319600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a34528-5921-45f4-a709-c6582bc9e41d None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.320130 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a34528-5921-45f4-a709-c6582bc9e41d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.320605 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a34528-5921-45f4-a709-c6582bc9e41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.387495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-67a34528-5921-45f4-a709-c6582bc9e41d None None] MFA Rules not processed for user `ec846d9fc63b4e9781949f70c7bf13fa`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:08.388794 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-67a34528-5921-45f4-a709-c6582bc9e41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:08.544246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 890/1755] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:08.556059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a249e3-972b-4987-b809-82adcc54d9c7 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.558191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b2a249e3-972b-4987-b809-82adcc54d9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:08.670459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a249e3-972b-4987-b809-82adcc54d9c7 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.680186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab2103-13d8-494f-98ae-7b380281e2f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.688363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab2103-13d8-494f-98ae-7b380281e2f4 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.688730 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab2103-13d8-494f-98ae-7b380281e2f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.689071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab2103-13d8-494f-98ae-7b380281e2f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.717683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab2103-13d8-494f-98ae-7b380281e2f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.721176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab2103-13d8-494f-98ae-7b380281e2f4 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.741278 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 866/1756] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:08.766536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a34528-5921-45f4-a709-c6582bc9e41d None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.866009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a34528-5921-45f4-a709-c6582bc9e41d None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.876102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.876780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.877059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.877362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] PATH_INFO: `/v3/users/ec846d9fc63b4e9781949f70c7bf13fa` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.879303 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec846d9fc63b4e9781949f70c7bf13fa)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.880980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.950386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec846d9fc63b4e9781949f70c7bf13fa'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:08.972852 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.975136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec846d9fc63b4e9781949f70c7bf13fa'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:08.989869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec846d9fc63b4e9781949f70c7bf13fa'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:08.991061 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec846d9fc63b4e9781949f70c7bf13fa'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:08.991883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec846d9fc63b4e9781949f70c7bf13fa'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:09.041271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 891/1757] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:08 2022] DELETE /identity/v3/users/ec846d9fc63b4e9781949f70c7bf13fa => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:09.043586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff82f08-4869-4e07-b27f-2c353ba98f21 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.044848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff82f08-4869-4e07-b27f-2c353ba98f21 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.045346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff82f08-4869-4e07-b27f-2c353ba98f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.070999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-dff82f08-4869-4e07-b27f-2c353ba98f21 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:09.072806 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-dff82f08-4869-4e07-b27f-2c353ba98f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:09.195995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 867/1758] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:09.212656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.233381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:09.454744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff795e-2cf4-492d-8fc8-abd9fb51ca4c admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.463743 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9d506-a788-423a-97ec-c32e9bdec70c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.464637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9d506-a788-423a-97ec-c32e9bdec70c service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.465097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9d506-a788-423a-97ec-c32e9bdec70c service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.465522 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9d506-a788-423a-97ec-c32e9bdec70c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.493882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9d506-a788-423a-97ec-c32e9bdec70c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.496411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9d506-a788-423a-97ec-c32e9bdec70c service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.528124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 892/1759] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:11.185331 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18329c89-d037-4e20-a6ba-f0f1d7580564 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.188572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18329c89-d037-4e20-a6ba-f0f1d7580564 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.189008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18329c89-d037-4e20-a6ba-f0f1d7580564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.275137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-18329c89-d037-4e20-a6ba-f0f1d7580564 None None] MFA Rules not processed for user `80097afc3ce540e6a8060f696535e62c`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:11.278723 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-18329c89-d037-4e20-a6ba-f0f1d7580564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:11.309241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 868/1760] 10.222.0.79 () {58 vars in 1015 bytes} [Fri Aug 5 19:50:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:11.468775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9d506-a788-423a-97ec-c32e9bdec70c service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:11.471837 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7cc9d506-a788-423a-97ec-c32e9bdec70c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:11.505650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9d506-a788-423a-97ec-c32e9bdec70c service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:11.512871 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:11.513535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.513815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.514086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f admin admin] PATH_INFO: `/v3/projects/02f03bc0fa114c54afc2a696fa8bd653` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.515425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f admin admin] RBAC: Authorizing `identity:delete_project(project_id=02f03bc0fa114c54afc2a696fa8bd653)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:11.518272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:11.664408 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02f03bc0fa114c54afc2a696fa8bd653', 'request_id': 'req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:11.685247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02f03bc0fa114c54afc2a696fa8bd653', 'request_id': 'req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:11.716667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02f03bc0fa114c54afc2a696fa8bd653', 'request_id': 'req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:11.746515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f admin admin] The token cache is being invalidate because project 02f03bc0fa114c54afc2a696fa8bd653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:11.747104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:11.748168 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3f9aacaa-8ea6-4b5c-a2db-2e97aa70c92f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:11.750499 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 893/1761] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:11 2022] DELETE /identity/v3/projects/02f03bc0fa114c54afc2a696fa8bd653 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:11.758196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e00b50-463b-472c-9220-95fc6b5a1088 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.758570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e00b50-463b-472c-9220-95fc6b5a1088 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.762996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e00b50-463b-472c-9220-95fc6b5a1088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.802039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-90e00b50-463b-472c-9220-95fc6b5a1088 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:11.810057 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-90e00b50-463b-472c-9220-95fc6b5a1088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:11.890609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:11.910855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 869/1762] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:11.916507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ac6a0-607c-43a3-86b5-201f98f6603c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.917785 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ac6a0-607c-43a3-86b5-201f98f6603c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.917785 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ac6a0-607c-43a3-86b5-201f98f6603c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.939906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3c4ac6a0-607c-43a3-86b5-201f98f6603c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:11.941367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3c4ac6a0-607c-43a3-86b5-201f98f6603c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:11.956392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 894/1763] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:11.960848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e00b50-463b-472c-9220-95fc6b5a1088 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:11.977852 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e00b50-463b-472c-9220-95fc6b5a1088 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:11.986520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb982bc-8b08-438d-832a-e6761bfd07d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:11.987853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb982bc-8b08-438d-832a-e6761bfd07d3 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.987853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb982bc-8b08-438d-832a-e6761bfd07d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.987853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb982bc-8b08-438d-832a-e6761bfd07d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.989117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb982bc-8b08-438d-832a-e6761bfd07d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:11.989573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb982bc-8b08-438d-832a-e6761bfd07d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:11.992057 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb982bc-8b08-438d-832a-e6761bfd07d3 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:12.034233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 870/1764] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:12.037392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ac6a0-607c-43a3-86b5-201f98f6603c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:12.049379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ac6a0-607c-43a3-86b5-201f98f6603c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:12.055456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77a5da-4692-435e-8cda-90e2dbf7a43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:12.055982 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77a5da-4692-435e-8cda-90e2dbf7a43d admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.056358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77a5da-4692-435e-8cda-90e2dbf7a43d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.056979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77a5da-4692-435e-8cda-90e2dbf7a43d admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.058057 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77a5da-4692-435e-8cda-90e2dbf7a43d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:12.060793 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77a5da-4692-435e-8cda-90e2dbf7a43d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:12.113755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 895/1765] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:12 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:12.115001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb982bc-8b08-438d-832a-e6761bfd07d3 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:12.139543 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb982bc-8b08-438d-832a-e6761bfd07d3 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:12.210571 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95628cb6-6fb8-46b9-bcc1-f0e3cdd9a7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:12.211293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95628cb6-6fb8-46b9-bcc1-f0e3cdd9a7bd admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.211497 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95628cb6-6fb8-46b9-bcc1-f0e3cdd9a7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.211770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95628cb6-6fb8-46b9-bcc1-f0e3cdd9a7bd admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.212435 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95628cb6-6fb8-46b9-bcc1-f0e3cdd9a7bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:12.215426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95628cb6-6fb8-46b9-bcc1-f0e3cdd9a7bd admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:12.242546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 871/1766] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:12 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:12.243871 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77a5da-4692-435e-8cda-90e2dbf7a43d admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:12.254506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77a5da-4692-435e-8cda-90e2dbf7a43d admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:12.260001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba47273-c1fd-4f97-aa1f-9deaa8b5e249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:12.261575 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba47273-c1fd-4f97-aa1f-9deaa8b5e249 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.261853 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba47273-c1fd-4f97-aa1f-9deaa8b5e249 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.262174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba47273-c1fd-4f97-aa1f-9deaa8b5e249 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.262969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba47273-c1fd-4f97-aa1f-9deaa8b5e249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:12.265483 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba47273-c1fd-4f97-aa1f-9deaa8b5e249 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:12.277648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 896/1767] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:12.282110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95628cb6-6fb8-46b9-bcc1-f0e3cdd9a7bd admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:12.292996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95628cb6-6fb8-46b9-bcc1-f0e3cdd9a7bd admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:12.298575 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6eb25-1950-492d-bea0-1cb1f19e3c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:12.299236 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6eb25-1950-492d-bea0-1cb1f19e3c39 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.299516 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6eb25-1950-492d-bea0-1cb1f19e3c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.299786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6eb25-1950-492d-bea0-1cb1f19e3c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.300484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6eb25-1950-492d-bea0-1cb1f19e3c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:12.302842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6eb25-1950-492d-bea0-1cb1f19e3c39 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:12.315589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 872/1768] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:12.319678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba47273-c1fd-4f97-aa1f-9deaa8b5e249 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:12.330858 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba47273-c1fd-4f97-aa1f-9deaa8b5e249 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:12.335620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d76595-5ef1-44aa-a82b-25e7f2d798ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:12.336654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d76595-5ef1-44aa-a82b-25e7f2d798ce admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.337144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d76595-5ef1-44aa-a82b-25e7f2d798ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.337613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d76595-5ef1-44aa-a82b-25e7f2d798ce admin admin] PATH_INFO: `/v3/projects/d6138454b9a7453a88fbb7d4e3aa3b9e/users/313e777432894307b8e45b3cc27f37b5/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.356154 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d76595-5ef1-44aa-a82b-25e7f2d798ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6138454b9a7453a88fbb7d4e3aa3b9e, user_id=313e777432894307b8e45b3cc27f37b5, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:12.359079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d76595-5ef1-44aa-a82b-25e7f2d798ce admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:12.375153 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 897/1769] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:12 2022] PUT /identity/v3/projects/d6138454b9a7453a88fbb7d4e3aa3b9e/users/313e777432894307b8e45b3cc27f37b5/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:12.385934 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc450d0-9f3e-4c45-924e-ba5da07e5162 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.386385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc450d0-9f3e-4c45-924e-ba5da07e5162 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.386749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc450d0-9f3e-4c45-924e-ba5da07e5162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.416550 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-8fc450d0-9f3e-4c45-924e-ba5da07e5162 None None] MFA Rules not processed for user `313e777432894307b8e45b3cc27f37b5`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:12.419054 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-8fc450d0-9f3e-4c45-924e-ba5da07e5162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:12.512084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 873/1770] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:12.523665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d76595-5ef1-44aa-a82b-25e7f2d798ce admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:12.527447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-13d76595-5ef1-44aa-a82b-25e7f2d798ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:12.533364 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:12.634849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d76595-5ef1-44aa-a82b-25e7f2d798ce admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:12.643395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34f140-98b1-465f-9433-f1ff4d8bd343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:12.644303 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34f140-98b1-465f-9433-f1ff4d8bd343 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.644806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34f140-98b1-465f-9433-f1ff4d8bd343 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.645378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34f140-98b1-465f-9433-f1ff4d8bd343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.656325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34f140-98b1-465f-9433-f1ff4d8bd343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:12.658314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34f140-98b1-465f-9433-f1ff4d8bd343 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:12.670756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 898/1771] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:12.694618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc450d0-9f3e-4c45-924e-ba5da07e5162 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:12.865432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:12.911927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc450d0-9f3e-4c45-924e-ba5da07e5162 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:12.925071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:12.925933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.926400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.926871 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] PATH_INFO: `/v3/users/313e777432894307b8e45b3cc27f37b5` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.929693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] RBAC: Authorizing `identity:delete_user(user_id=313e777432894307b8e45b3cc27f37b5)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:12.934531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.056087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '313e777432894307b8e45b3cc27f37b5'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:13.065560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '313e777432894307b8e45b3cc27f37b5'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:13.066488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '313e777432894307b8e45b3cc27f37b5'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:13.085563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '313e777432894307b8e45b3cc27f37b5'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:13.108004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '313e777432894307b8e45b3cc27f37b5'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:13.120438 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 874/1772] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:12 2022] DELETE /identity/v3/users/313e777432894307b8e45b3cc27f37b5 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:13.124941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7259c-925f-4579-bbd7-9ca7bd4ffbea None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.125183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7259c-925f-4579-bbd7-9ca7bd4ffbea None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.125454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7259c-925f-4579-bbd7-9ca7bd4ffbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.157925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c2a7259c-925f-4579-bbd7-9ca7bd4ffbea None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:13.159505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c2a7259c-925f-4579-bbd7-9ca7bd4ffbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:13.305736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 899/1773] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:13.314199 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.315663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:13.420474 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0d0bf-2179-47de-868d-e87b82c4081f admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.425059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe91662-a414-491c-a3f8-604aacc26f6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.425683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91662-a414-491c-a3f8-604aacc26f6b service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.425963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91662-a414-491c-a3f8-604aacc26f6b service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.426234 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe91662-a414-491c-a3f8-604aacc26f6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.443675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe91662-a414-491c-a3f8-604aacc26f6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.445504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe91662-a414-491c-a3f8-604aacc26f6b service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.458790 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 875/1774] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:13.855147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7259c-925f-4579-bbd7-9ca7bd4ffbea None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.860257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c2a7259c-925f-4579-bbd7-9ca7bd4ffbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:13.931822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7259c-925f-4579-bbd7-9ca7bd4ffbea None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.957450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db582f14-144d-417e-ac61-a3634e891987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.961689 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db582f14-144d-417e-ac61-a3634e891987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.962085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db582f14-144d-417e-ac61-a3634e891987 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.962432 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db582f14-144d-417e-ac61-a3634e891987 admin admin] PATH_INFO: `/v3/projects/d6138454b9a7453a88fbb7d4e3aa3b9e` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.969000 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db582f14-144d-417e-ac61-a3634e891987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6138454b9a7453a88fbb7d4e3aa3b9e)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.972339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db582f14-144d-417e-ac61-a3634e891987 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.087637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-db582f14-144d-417e-ac61-a3634e891987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6138454b9a7453a88fbb7d4e3aa3b9e', 'request_id': 'req-db582f14-144d-417e-ac61-a3634e891987'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:14.106811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-db582f14-144d-417e-ac61-a3634e891987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6138454b9a7453a88fbb7d4e3aa3b9e', 'request_id': 'req-db582f14-144d-417e-ac61-a3634e891987'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:14.140069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-db582f14-144d-417e-ac61-a3634e891987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6138454b9a7453a88fbb7d4e3aa3b9e', 'request_id': 'req-db582f14-144d-417e-ac61-a3634e891987'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:14.171159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-db582f14-144d-417e-ac61-a3634e891987 admin admin] The token cache is being invalidate because project d6138454b9a7453a88fbb7d4e3aa3b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:14.171452 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-db582f14-144d-417e-ac61-a3634e891987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:14.172684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-db582f14-144d-417e-ac61-a3634e891987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:14.178980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 900/1775] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:13 2022] DELETE /identity/v3/projects/d6138454b9a7453a88fbb7d4e3aa3b9e => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:14.180775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b8454-7ec1-430f-8734-360bf74f5003 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.181069 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b8454-7ec1-430f-8734-360bf74f5003 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.181343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b8454-7ec1-430f-8734-360bf74f5003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.237712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-1a1b8454-7ec1-430f-8734-360bf74f5003 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:14.239007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-1a1b8454-7ec1-430f-8734-360bf74f5003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:14.373504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 876/1776] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:14.376341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7967561-b283-4a32-af78-113a501f1c35 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.376848 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7967561-b283-4a32-af78-113a501f1c35 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.377295 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7967561-b283-4a32-af78-113a501f1c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.405690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e7967561-b283-4a32-af78-113a501f1c35 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:14.407775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e7967561-b283-4a32-af78-113a501f1c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:14.423033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 901/1777] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:14.427579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b8454-7ec1-430f-8734-360bf74f5003 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.439697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b8454-7ec1-430f-8734-360bf74f5003 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.446722 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f9b2e-3320-47a1-8864-8beaef37bdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.448309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f9b2e-3320-47a1-8864-8beaef37bdf9 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.448764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f9b2e-3320-47a1-8864-8beaef37bdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.449380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f9b2e-3320-47a1-8864-8beaef37bdf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.450811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f9b2e-3320-47a1-8864-8beaef37bdf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:14.451911 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f9b2e-3320-47a1-8864-8beaef37bdf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.454247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f9b2e-3320-47a1-8864-8beaef37bdf9 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.478938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 877/1778] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:14.483914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7967561-b283-4a32-af78-113a501f1c35 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.498511 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7967561-b283-4a32-af78-113a501f1c35 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.515870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7e8f8-e012-458f-a6fc-9f983729e8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.515870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7e8f8-e012-458f-a6fc-9f983729e8a1 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.515870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7e8f8-e012-458f-a6fc-9f983729e8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.515870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7e8f8-e012-458f-a6fc-9f983729e8a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.516260 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7e8f8-e012-458f-a6fc-9f983729e8a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.518390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7e8f8-e012-458f-a6fc-9f983729e8a1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.556843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.570779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 902/1779] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:14 2022] POST /identity/v3/projects => generated 359 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:14.572296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f9b2e-3320-47a1-8864-8beaef37bdf9 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.581026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.596896 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f9b2e-3320-47a1-8864-8beaef37bdf9 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.676762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299f2d6-47a7-431e-8f19-feb711e0989e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.677392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299f2d6-47a7-431e-8f19-feb711e0989e admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.677666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299f2d6-47a7-431e-8f19-feb711e0989e admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.677933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299f2d6-47a7-431e-8f19-feb711e0989e admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.678751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299f2d6-47a7-431e-8f19-feb711e0989e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.680742 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299f2d6-47a7-431e-8f19-feb711e0989e admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.713118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 878/1780] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:14 2022] POST /identity/v3/users => generated 280 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:14.717265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7e8f8-e012-458f-a6fc-9f983729e8a1 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.728891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7e8f8-e012-458f-a6fc-9f983729e8a1 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.738572 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41063305-4f63-4b58-b8da-60e90f3b4ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.739388 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41063305-4f63-4b58-b8da-60e90f3b4ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.739672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41063305-4f63-4b58-b8da-60e90f3b4ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.739978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41063305-4f63-4b58-b8da-60e90f3b4ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.740765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41063305-4f63-4b58-b8da-60e90f3b4ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.742406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41063305-4f63-4b58-b8da-60e90f3b4ac8 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.757823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 903/1781] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:14.763090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299f2d6-47a7-431e-8f19-feb711e0989e admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.778745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299f2d6-47a7-431e-8f19-feb711e0989e admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.784089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18112f-86bc-4f57-b63d-bc20c98329c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.784837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18112f-86bc-4f57-b63d-bc20c98329c6 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.785220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18112f-86bc-4f57-b63d-bc20c98329c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.785584 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18112f-86bc-4f57-b63d-bc20c98329c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.786383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18112f-86bc-4f57-b63d-bc20c98329c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.788116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18112f-86bc-4f57-b63d-bc20c98329c6 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.800577 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 879/1782] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:14.804978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41063305-4f63-4b58-b8da-60e90f3b4ac8 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.817117 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41063305-4f63-4b58-b8da-60e90f3b4ac8 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.831255 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a8f05-ae6e-445e-9782-f28d942415a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.832232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a8f05-ae6e-445e-9782-f28d942415a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.832511 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a8f05-ae6e-445e-9782-f28d942415a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.832855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a8f05-ae6e-445e-9782-f28d942415a5 admin admin] PATH_INFO: `/v3/projects/57e9d0b2349841589edb85c01811a269/users/8df03015d991404d9c9ba4d648d1876e/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.849954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a8f05-ae6e-445e-9782-f28d942415a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57e9d0b2349841589edb85c01811a269, user_id=8df03015d991404d9c9ba4d648d1876e, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.852894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a8f05-ae6e-445e-9782-f28d942415a5 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.874797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 904/1783] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:14 2022] PUT /identity/v3/projects/57e9d0b2349841589edb85c01811a269/users/8df03015d991404d9c9ba4d648d1876e/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:14.882883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da144c-6a00-43a2-860d-d64945d95acb None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.883164 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da144c-6a00-43a2-860d-d64945d95acb None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.883437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da144c-6a00-43a2-860d-d64945d95acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.923205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-d1da144c-6a00-43a2-860d-d64945d95acb None None] MFA Rules not processed for user `8df03015d991404d9c9ba4d648d1876e`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:14.924365 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d1da144c-6a00-43a2-860d-d64945d95acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:15.001174 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 880/1784] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:15.039376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a8f05-ae6e-445e-9782-f28d942415a5 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.041145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-bc1a8f05-ae6e-445e-9782-f28d942415a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:15.114112 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a8f05-ae6e-445e-9782-f28d942415a5 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.118224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f095c6b-c211-4b12-b2bb-bc8208e8376a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.118889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f095c6b-c211-4b12-b2bb-bc8208e8376a service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.119171 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f095c6b-c211-4b12-b2bb-bc8208e8376a service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.119448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f095c6b-c211-4b12-b2bb-bc8208e8376a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.129858 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f095c6b-c211-4b12-b2bb-bc8208e8376a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.131344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f095c6b-c211-4b12-b2bb-bc8208e8376a service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.145652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 905/1785] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:15.172106 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da144c-6a00-43a2-860d-d64945d95acb None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.309140 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da144c-6a00-43a2-860d-d64945d95acb None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.319956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.319956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.319956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.319956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] PATH_INFO: `/v3/users/8df03015d991404d9c9ba4d648d1876e` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.324670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df03015d991404d9c9ba4d648d1876e)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.327448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.357972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.439122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df03015d991404d9c9ba4d648d1876e'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:15.448733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df03015d991404d9c9ba4d648d1876e'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:15.451108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df03015d991404d9c9ba4d648d1876e'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:15.467154 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df03015d991404d9c9ba4d648d1876e'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:15.484560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df03015d991404d9c9ba4d648d1876e'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:15.508457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 881/1786] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:15 2022] DELETE /identity/v3/users/8df03015d991404d9c9ba4d648d1876e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:15.513600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab071fcc-d32e-4b3b-a208-95910075bf69 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.514396 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab071fcc-d32e-4b3b-a208-95910075bf69 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.514900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab071fcc-d32e-4b3b-a208-95910075bf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.550874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-ab071fcc-d32e-4b3b-a208-95910075bf69 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:15.558445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ab071fcc-d32e-4b3b-a208-95910075bf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:15.636016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 906/1787] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:15.645909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.647517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:15.708522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8affcf-6109-4227-8f31-8df932a74171 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.712065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c61eb7-6fa2-4823-a982-9e8b75992261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.712673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c61eb7-6fa2-4823-a982-9e8b75992261 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.712946 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c61eb7-6fa2-4823-a982-9e8b75992261 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.713213 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c61eb7-6fa2-4823-a982-9e8b75992261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.723629 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c61eb7-6fa2-4823-a982-9e8b75992261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.725046 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c61eb7-6fa2-4823-a982-9e8b75992261 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.735837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 882/1788] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:16.107033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab071fcc-d32e-4b3b-a208-95910075bf69 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.109101 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ab071fcc-d32e-4b3b-a208-95910075bf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:16.125736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab071fcc-d32e-4b3b-a208-95910075bf69 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.131742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50909182-a0b5-4980-8083-652f38d1c5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.132431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50909182-a0b5-4980-8083-652f38d1c5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.132691 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50909182-a0b5-4980-8083-652f38d1c5df admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.132963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50909182-a0b5-4980-8083-652f38d1c5df admin admin] PATH_INFO: `/v3/projects/57e9d0b2349841589edb85c01811a269` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.134802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50909182-a0b5-4980-8083-652f38d1c5df admin admin] RBAC: Authorizing `identity:delete_project(project_id=57e9d0b2349841589edb85c01811a269)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.138126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50909182-a0b5-4980-8083-652f38d1c5df admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.200995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-50909182-a0b5-4980-8083-652f38d1c5df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57e9d0b2349841589edb85c01811a269', 'request_id': 'req-50909182-a0b5-4980-8083-652f38d1c5df'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:16.208085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-50909182-a0b5-4980-8083-652f38d1c5df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57e9d0b2349841589edb85c01811a269', 'request_id': 'req-50909182-a0b5-4980-8083-652f38d1c5df'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:16.218104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-50909182-a0b5-4980-8083-652f38d1c5df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57e9d0b2349841589edb85c01811a269', 'request_id': 'req-50909182-a0b5-4980-8083-652f38d1c5df'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:16.224162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.252573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-50909182-a0b5-4980-8083-652f38d1c5df admin admin] The token cache is being invalidate because project 57e9d0b2349841589edb85c01811a269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:16.252573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-50909182-a0b5-4980-8083-652f38d1c5df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:16.252573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-50909182-a0b5-4980-8083-652f38d1c5df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:16.252573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 907/1789] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:16 2022] DELETE /identity/v3/projects/57e9d0b2349841589edb85c01811a269 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:16.253111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82a073-cd82-4f3b-8bc1-d56a69461c49 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.253111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82a073-cd82-4f3b-8bc1-d56a69461c49 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.253111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82a073-cd82-4f3b-8bc1-d56a69461c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.270148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-ec82a073-cd82-4f3b-8bc1-d56a69461c49 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:16.272041 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ec82a073-cd82-4f3b-8bc1-d56a69461c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:16.436583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 883/1790] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:16.445342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1093d33-028d-4026-9f4e-41dbe2c1536a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.445673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1093d33-028d-4026-9f4e-41dbe2c1536a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.446136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1093d33-028d-4026-9f4e-41dbe2c1536a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.508940 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-b1093d33-028d-4026-9f4e-41dbe2c1536a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:16.511113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b1093d33-028d-4026-9f4e-41dbe2c1536a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:16.532899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 908/1791] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:16.533939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82a073-cd82-4f3b-8bc1-d56a69461c49 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.545225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82a073-cd82-4f3b-8bc1-d56a69461c49 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.561349 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60977ca5-f108-416b-92f7-94dedf849465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.562018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60977ca5-f108-416b-92f7-94dedf849465 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.562298 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60977ca5-f108-416b-92f7-94dedf849465 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.562729 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60977ca5-f108-416b-92f7-94dedf849465 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.563461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60977ca5-f108-416b-92f7-94dedf849465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:16.563774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60977ca5-f108-416b-92f7-94dedf849465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.583089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60977ca5-f108-416b-92f7-94dedf849465 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.628083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 884/1792] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:16.628609 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1093d33-028d-4026-9f4e-41dbe2c1536a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.643194 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1093d33-028d-4026-9f4e-41dbe2c1536a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.652111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04354495-428b-4fb2-9670-5b9f0a383761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.670767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04354495-428b-4fb2-9670-5b9f0a383761 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.671140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04354495-428b-4fb2-9670-5b9f0a383761 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.671500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04354495-428b-4fb2-9670-5b9f0a383761 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.672279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04354495-428b-4fb2-9670-5b9f0a383761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.674449 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04354495-428b-4fb2-9670-5b9f0a383761 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.761841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 909/1793] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:16 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:16.763525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60977ca5-f108-416b-92f7-94dedf849465 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.777113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60977ca5-f108-416b-92f7-94dedf849465 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.841765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.981386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01d9d2-9331-4bd8-bb88-1d888d7ba0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.983389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01d9d2-9331-4bd8-bb88-1d888d7ba0fc admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.984181 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01d9d2-9331-4bd8-bb88-1d888d7ba0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.984533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01d9d2-9331-4bd8-bb88-1d888d7ba0fc admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.986439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01d9d2-9331-4bd8-bb88-1d888d7ba0fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.997880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01d9d2-9331-4bd8-bb88-1d888d7ba0fc admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.049946 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 885/1794] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:16 2022] POST /identity/v3/users => generated 294 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:17.055122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04354495-428b-4fb2-9670-5b9f0a383761 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.107812 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04354495-428b-4fb2-9670-5b9f0a383761 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.117173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ec5db-72be-432e-af91-ac341269b7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.117973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ec5db-72be-432e-af91-ac341269b7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.121573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ec5db-72be-432e-af91-ac341269b7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.122059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ec5db-72be-432e-af91-ac341269b7c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.123101 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ec5db-72be-432e-af91-ac341269b7c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.140016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ec5db-72be-432e-af91-ac341269b7c4 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.159717 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01d9d2-9331-4bd8-bb88-1d888d7ba0fc admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.162370 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-de01d9d2-9331-4bd8-bb88-1d888d7ba0fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:17.174436 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 910/1795] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:17.178912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ec5db-72be-432e-af91-ac341269b7c4 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.195775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ec5db-72be-432e-af91-ac341269b7c4 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.210688 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181e548-6d88-4aa6-b8b3-5e34ae13cff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.211560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181e548-6d88-4aa6-b8b3-5e34ae13cff1 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.211998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181e548-6d88-4aa6-b8b3-5e34ae13cff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.214611 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181e548-6d88-4aa6-b8b3-5e34ae13cff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.216264 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181e548-6d88-4aa6-b8b3-5e34ae13cff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.219278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181e548-6d88-4aa6-b8b3-5e34ae13cff1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.254859 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 911/1796] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:17.280461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181e548-6d88-4aa6-b8b3-5e34ae13cff1 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.326145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01d9d2-9331-4bd8-bb88-1d888d7ba0fc admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.329102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181e548-6d88-4aa6-b8b3-5e34ae13cff1 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.331270 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0bef8-00c7-4920-953b-9f6208124cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.332325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0bef8-00c7-4920-953b-9f6208124cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.332808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0bef8-00c7-4920-953b-9f6208124cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.333326 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0bef8-00c7-4920-953b-9f6208124cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.336002 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d6f0bef8-00c7-4920-953b-9f6208124cc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:17.352258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50d03c-2904-4e8d-a3bf-0b86bf47deac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.354902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0bef8-00c7-4920-953b-9f6208124cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.356809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50d03c-2904-4e8d-a3bf-0b86bf47deac admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.357107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50d03c-2904-4e8d-a3bf-0b86bf47deac admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.357238 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0bef8-00c7-4920-953b-9f6208124cc2 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.357388 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50d03c-2904-4e8d-a3bf-0b86bf47deac admin admin] PATH_INFO: `/v3/projects/669187f32db34f59ac18dc23a4cbd8a3/users/399994deb42e46e399e82d3db4fa7876/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.380992 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50d03c-2904-4e8d-a3bf-0b86bf47deac admin admin] RBAC: Authorizing `identity:create_grant(project_id=669187f32db34f59ac18dc23a4cbd8a3, user_id=399994deb42e46e399e82d3db4fa7876, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.381546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 886/1797] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:50:17.384755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50d03c-2904-4e8d-a3bf-0b86bf47deac admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.406559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 912/1798] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:17 2022] PUT /identity/v3/projects/669187f32db34f59ac18dc23a4cbd8a3/users/399994deb42e46e399e82d3db4fa7876/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:17.427825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ce04c-6f85-404b-b928-d0958504100a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.428399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ce04c-6f85-404b-b928-d0958504100a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.428860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ce04c-6f85-404b-b928-d0958504100a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.520120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-093ce04c-6f85-404b-b928-d0958504100a None None] MFA Rules not processed for user `399994deb42e46e399e82d3db4fa7876`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:17.521277 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-093ce04c-6f85-404b-b928-d0958504100a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:17.746962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 887/1799] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:17.772734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50d03c-2904-4e8d-a3bf-0b86bf47deac admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.774543 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3f50d03c-2904-4e8d-a3bf-0b86bf47deac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:17.907821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50d03c-2904-4e8d-a3bf-0b86bf47deac admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.915531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296221a-7e5d-4510-8564-0703ebd2c60a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.917098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296221a-7e5d-4510-8564-0703ebd2c60a service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.917619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296221a-7e5d-4510-8564-0703ebd2c60a service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.918279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296221a-7e5d-4510-8564-0703ebd2c60a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.943499 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296221a-7e5d-4510-8564-0703ebd2c60a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.944980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296221a-7e5d-4510-8564-0703ebd2c60a service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.967148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 913/1800] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:18.901796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2392571-5fae-441e-8f4c-be0aff7f7fbb None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.902510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2392571-5fae-441e-8f4c-be0aff7f7fbb None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.902595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2392571-5fae-441e-8f4c-be0aff7f7fbb None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.905513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 888/1801] 10.222.0.86 () {54 vars in 921 bytes} [Fri Aug 5 19:50:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:18.922757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff25e2a-be08-4136-91bc-5619ecae115b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.923388 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff25e2a-be08-4136-91bc-5619ecae115b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.923388 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff25e2a-be08-4136-91bc-5619ecae115b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.961879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-6ff25e2a-be08-4136-91bc-5619ecae115b None None] MFA Rules not processed for user `ab8f9d9cf4ca4e38ae6eb67e949b40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:18.963120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6ff25e2a-be08-4136-91bc-5619ecae115b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:19.181332 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 914/1802] 10.222.0.86 () {58 vars in 1018 bytes} [Fri Aug 5 19:50:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:19.202848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2392571-5fae-441e-8f4c-be0aff7f7fbb None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.205768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e2392571-5fae-441e-8f4c-be0aff7f7fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:19.233546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2392571-5fae-441e-8f4c-be0aff7f7fbb None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.239610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a359f-d53c-4786-8672-b53e627de297 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.240865 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a359f-d53c-4786-8672-b53e627de297 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.241313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a359f-d53c-4786-8672-b53e627de297 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.241758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a359f-d53c-4786-8672-b53e627de297 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.259272 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a359f-d53c-4786-8672-b53e627de297 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.262306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a359f-d53c-4786-8672-b53e627de297 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.283577 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 889/1803] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:20.528860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff25e2a-be08-4136-91bc-5619ecae115b None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.530261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6ff25e2a-be08-4136-91bc-5619ecae115b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:20.552576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.607699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff25e2a-be08-4136-91bc-5619ecae115b None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.612166 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.612805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.613110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.613381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] PATH_INFO: `/v3/users/ffbb0c59812e412ba17a15a28104aa63` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.614535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffbb0c59812e412ba17a15a28104aa63)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.616617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.666142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffbb0c59812e412ba17a15a28104aa63'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:20.679940 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffbb0c59812e412ba17a15a28104aa63'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:20.687661 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffbb0c59812e412ba17a15a28104aa63'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:20.688422 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffbb0c59812e412ba17a15a28104aa63'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:20.689135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffbb0c59812e412ba17a15a28104aa63'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:20.703783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 915/1804] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:20 2022] DELETE /identity/v3/users/ffbb0c59812e412ba17a15a28104aa63 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:20.708785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a622f8a-9ede-4fdc-94bd-82b5813f95a5 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.709310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a622f8a-9ede-4fdc-94bd-82b5813f95a5 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.709475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a622f8a-9ede-4fdc-94bd-82b5813f95a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.730348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-0a622f8a-9ede-4fdc-94bd-82b5813f95a5 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:20.731593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-0a622f8a-9ede-4fdc-94bd-82b5813f95a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:20.774889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.813914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 890/1805] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:20.824363 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.826425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:20.889910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ff136-753b-46f7-8f36-a5fd0c351d16 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.897577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d1009-bf86-4e78-9b9a-e8c796b04de8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.898225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d1009-bf86-4e78-9b9a-e8c796b04de8 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.898507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d1009-bf86-4e78-9b9a-e8c796b04de8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.898846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d1009-bf86-4e78-9b9a-e8c796b04de8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.909982 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d1009-bf86-4e78-9b9a-e8c796b04de8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.911472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d1009-bf86-4e78-9b9a-e8c796b04de8 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.922090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 916/1806] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:50:21.211802 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a622f8a-9ede-4fdc-94bd-82b5813f95a5 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.213427 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-0a622f8a-9ede-4fdc-94bd-82b5813f95a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:21.245488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a622f8a-9ede-4fdc-94bd-82b5813f95a5 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.252329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.263657 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.264155 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.264426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] PATH_INFO: `/v3/projects/7558ffaf30d44c37b8d2c4a7807d88f9` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.265822 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7558ffaf30d44c37b8d2c4a7807d88f9)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.268111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.341350 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7558ffaf30d44c37b8d2c4a7807d88f9', 'request_id': 'req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:21.348065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7558ffaf30d44c37b8d2c4a7807d88f9', 'request_id': 'req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:21.359224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7558ffaf30d44c37b8d2c4a7807d88f9', 'request_id': 'req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:21.372781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] The token cache is being invalidate because project 7558ffaf30d44c37b8d2c4a7807d88f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:21.373184 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:21.374075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:21.376108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 891/1807] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:21 2022] DELETE /identity/v3/projects/7558ffaf30d44c37b8d2c4a7807d88f9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:21.380044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d1009-bf86-4e78-9b9a-e8c796b04de8 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.389935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d1009-bf86-4e78-9b9a-e8c796b04de8 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.421043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:21.456122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48053a32-432b-4d63-98c9-91a08b8adf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.458405 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48053a32-432b-4d63-98c9-91a08b8adf33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.458830 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48053a32-432b-4d63-98c9-91a08b8adf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.459287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48053a32-432b-4d63-98c9-91a08b8adf33 admin admin] PATH_INFO: `/v3/users/b2e73b3d830b437388c236835112d007` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.460893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48053a32-432b-4d63-98c9-91a08b8adf33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e73b3d830b437388c236835112d007)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.463089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48053a32-432b-4d63-98c9-91a08b8adf33 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.532453 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-48053a32-432b-4d63-98c9-91a08b8adf33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e73b3d830b437388c236835112d007'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:21.549223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-48053a32-432b-4d63-98c9-91a08b8adf33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e73b3d830b437388c236835112d007'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:21.557841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-48053a32-432b-4d63-98c9-91a08b8adf33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e73b3d830b437388c236835112d007'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:21.558728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-48053a32-432b-4d63-98c9-91a08b8adf33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e73b3d830b437388c236835112d007'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:21.559374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-48053a32-432b-4d63-98c9-91a08b8adf33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e73b3d830b437388c236835112d007'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:21.574893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 917/1808] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:21 2022] DELETE /identity/v3/users/b2e73b3d830b437388c236835112d007 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:21.871751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.873279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:21.887754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d21ee6-c4d4-4073-8b0d-5c0c02fad3fe admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.980034 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.980764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.981040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.981303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081 admin admin] PATH_INFO: `/v3/projects/54b3d9e6545e4efa889d567e0a40f173` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.982585 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54b3d9e6545e4efa889d567e0a40f173)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.986274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.049983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54b3d9e6545e4efa889d567e0a40f173', 'request_id': 'req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:22.061817 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54b3d9e6545e4efa889d567e0a40f173', 'request_id': 'req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:22.063571 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:22.070360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54b3d9e6545e4efa889d567e0a40f173', 'request_id': 'req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:22.090268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081 admin admin] The token cache is being invalidate because project 54b3d9e6545e4efa889d567e0a40f173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:22.090268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:22.090268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ab53dfaf-8ce5-4b34-ba51-6ea32a741081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:22.090268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 892/1809] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:21 2022] DELETE /identity/v3/projects/54b3d9e6545e4efa889d567e0a40f173 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:22.091832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f838032-8db5-4063-86ac-9acdb40eaf62 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.092114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f838032-8db5-4063-86ac-9acdb40eaf62 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.092382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f838032-8db5-4063-86ac-9acdb40eaf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.113721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-9f838032-8db5-4063-86ac-9acdb40eaf62 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:22.115022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9f838032-8db5-4063-86ac-9acdb40eaf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:22.178096 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 918/1810] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:22.185003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e8020-dd33-4ffe-a036-526e2a3471cb None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.185284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e8020-dd33-4ffe-a036-526e2a3471cb None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.185556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e8020-dd33-4ffe-a036-526e2a3471cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.206766 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-bb0e8020-dd33-4ffe-a036-526e2a3471cb None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:22.207968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-bb0e8020-dd33-4ffe-a036-526e2a3471cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:22.216877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 893/1811] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:22.221250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f838032-8db5-4063-86ac-9acdb40eaf62 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.231371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f838032-8db5-4063-86ac-9acdb40eaf62 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.235800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befcae1-34d9-467a-a6f0-c8ed94a04176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.236408 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befcae1-34d9-467a-a6f0-c8ed94a04176 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.236679 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befcae1-34d9-467a-a6f0-c8ed94a04176 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.236943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befcae1-34d9-467a-a6f0-c8ed94a04176 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.237632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befcae1-34d9-467a-a6f0-c8ed94a04176 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:22.237948 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befcae1-34d9-467a-a6f0-c8ed94a04176 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.239585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befcae1-34d9-467a-a6f0-c8ed94a04176 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.252697 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 919/1812] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:22.256956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e8020-dd33-4ffe-a036-526e2a3471cb None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.266808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e8020-dd33-4ffe-a036-526e2a3471cb None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.271354 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9976d6f-f6f3-493f-8e05-0f508008a082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.271950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9976d6f-f6f3-493f-8e05-0f508008a082 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.272223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9976d6f-f6f3-493f-8e05-0f508008a082 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.272487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9976d6f-f6f3-493f-8e05-0f508008a082 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.273138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9976d6f-f6f3-493f-8e05-0f508008a082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.275047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9976d6f-f6f3-493f-8e05-0f508008a082 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.303245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 894/1813] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:22 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:22.308453 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befcae1-34d9-467a-a6f0-c8ed94a04176 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.319055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befcae1-34d9-467a-a6f0-c8ed94a04176 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.361587 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:22.394565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43e765-2234-4ec5-aa2b-f6228e65d290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.394565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43e765-2234-4ec5-aa2b-f6228e65d290 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.394974 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43e765-2234-4ec5-aa2b-f6228e65d290 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.395103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43e765-2234-4ec5-aa2b-f6228e65d290 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.395831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43e765-2234-4ec5-aa2b-f6228e65d290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.397787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43e765-2234-4ec5-aa2b-f6228e65d290 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.433726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 920/1814] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:22 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:22.438718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9976d6f-f6f3-493f-8e05-0f508008a082 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.450376 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9976d6f-f6f3-493f-8e05-0f508008a082 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.455099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64e03c-7632-4a56-a94f-5d4407f75c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.455704 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64e03c-7632-4a56-a94f-5d4407f75c68 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.455976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64e03c-7632-4a56-a94f-5d4407f75c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.456243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64e03c-7632-4a56-a94f-5d4407f75c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.456931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64e03c-7632-4a56-a94f-5d4407f75c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.458500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64e03c-7632-4a56-a94f-5d4407f75c68 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.468877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 895/1815] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:22.473046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43e765-2234-4ec5-aa2b-f6228e65d290 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.483652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43e765-2234-4ec5-aa2b-f6228e65d290 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.488231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb23b5-b119-4807-bec2-fba0e35ad3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.488837 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb23b5-b119-4807-bec2-fba0e35ad3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.489115 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb23b5-b119-4807-bec2-fba0e35ad3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.489385 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb23b5-b119-4807-bec2-fba0e35ad3e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.490074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb23b5-b119-4807-bec2-fba0e35ad3e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.491704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb23b5-b119-4807-bec2-fba0e35ad3e2 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.502492 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 921/1816] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:22.506745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64e03c-7632-4a56-a94f-5d4407f75c68 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.517927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64e03c-7632-4a56-a94f-5d4407f75c68 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.522721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c46bb-29d2-4df4-b549-5074a8db6c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.523382 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c46bb-29d2-4df4-b549-5074a8db6c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.523658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c46bb-29d2-4df4-b549-5074a8db6c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.523926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c46bb-29d2-4df4-b549-5074a8db6c78 admin admin] PATH_INFO: `/v3/projects/7351b72568704b5795091c260d2e6aca/users/d8e3738cbc714babbeb8742eeff97ec2/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.540225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c46bb-29d2-4df4-b549-5074a8db6c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7351b72568704b5795091c260d2e6aca, user_id=d8e3738cbc714babbeb8742eeff97ec2, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.543189 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c46bb-29d2-4df4-b549-5074a8db6c78 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.559749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 896/1817] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:22 2022] PUT /identity/v3/projects/7351b72568704b5795091c260d2e6aca/users/d8e3738cbc714babbeb8742eeff97ec2/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:22.570205 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37422953-c564-4385-b16d-e3b17f043ecc None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.570742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37422953-c564-4385-b16d-e3b17f043ecc None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.570842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37422953-c564-4385-b16d-e3b17f043ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.599819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-37422953-c564-4385-b16d-e3b17f043ecc None None] MFA Rules not processed for user `d8e3738cbc714babbeb8742eeff97ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:22.601045 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-37422953-c564-4385-b16d-e3b17f043ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:22.672235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 922/1818] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:22.676705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c46bb-29d2-4df4-b549-5074a8db6c78 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.740617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c46bb-29d2-4df4-b549-5074a8db6c78 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.745922 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9a551-86a2-4534-b53d-e1e7231167b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.746528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9a551-86a2-4534-b53d-e1e7231167b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.746848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9a551-86a2-4534-b53d-e1e7231167b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.747115 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9a551-86a2-4534-b53d-e1e7231167b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.747777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9a551-86a2-4534-b53d-e1e7231167b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.749652 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9a551-86a2-4534-b53d-e1e7231167b7 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.776189 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 897/1819] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:22 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:22.780373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37422953-c564-4385-b16d-e3b17f043ecc None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.792535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37422953-c564-4385-b16d-e3b17f043ecc None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.862124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360b759-643c-4bc9-83f9-5c64c924623b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.863390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360b759-643c-4bc9-83f9-5c64c924623b admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.863390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360b759-643c-4bc9-83f9-5c64c924623b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.875744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360b759-643c-4bc9-83f9-5c64c924623b admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.875744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360b759-643c-4bc9-83f9-5c64c924623b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.877781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360b759-643c-4bc9-83f9-5c64c924623b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.909132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 923/1820] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:22 2022] POST /identity/v3/users => generated 296 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:22.911855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9a551-86a2-4534-b53d-e1e7231167b7 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.925437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9a551-86a2-4534-b53d-e1e7231167b7 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.934873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6598ac-bd96-408f-979e-93bb30cd10a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.935937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6598ac-bd96-408f-979e-93bb30cd10a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.936255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6598ac-bd96-408f-979e-93bb30cd10a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.936533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6598ac-bd96-408f-979e-93bb30cd10a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.937344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6598ac-bd96-408f-979e-93bb30cd10a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.939456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6598ac-bd96-408f-979e-93bb30cd10a6 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.967834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 898/1821] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:22.973962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360b759-643c-4bc9-83f9-5c64c924623b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:23.003717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360b759-643c-4bc9-83f9-5c64c924623b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:23.026173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966278b-b2b7-4403-978d-92ac704a1e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:23.028480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966278b-b2b7-4403-978d-92ac704a1e53 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.029915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966278b-b2b7-4403-978d-92ac704a1e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.032133 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966278b-b2b7-4403-978d-92ac704a1e53 admin admin] PATH_INFO: `/v3/projects/1e239d8e525c4631a1eb8f4eede4bed0/users/a26946bb7485409f800e29f66d5e30f2/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.091819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966278b-b2b7-4403-978d-92ac704a1e53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e239d8e525c4631a1eb8f4eede4bed0, user_id=a26946bb7485409f800e29f66d5e30f2, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:23.108196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966278b-b2b7-4403-978d-92ac704a1e53 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:23.134240 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 924/1822] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:22 2022] PUT /identity/v3/projects/1e239d8e525c4631a1eb8f4eede4bed0/users/a26946bb7485409f800e29f66d5e30f2/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:23.141094 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6598ac-bd96-408f-979e-93bb30cd10a6 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:23.178014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6598ac-bd96-408f-979e-93bb30cd10a6 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:23.313012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:23.331799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef23793-1a67-4458-a43d-72a40f8f502f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:23.332929 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef23793-1a67-4458-a43d-72a40f8f502f admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.333528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef23793-1a67-4458-a43d-72a40f8f502f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.333982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef23793-1a67-4458-a43d-72a40f8f502f admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.335167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef23793-1a67-4458-a43d-72a40f8f502f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:23.340560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef23793-1a67-4458-a43d-72a40f8f502f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:23.363891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966278b-b2b7-4403-978d-92ac704a1e53 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:23.364696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 899/1823] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:23.377357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966278b-b2b7-4403-978d-92ac704a1e53 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:23.384392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067a11e-36b5-4ee7-b7e3-96fef15a2fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:23.385747 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067a11e-36b5-4ee7-b7e3-96fef15a2fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.386241 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067a11e-36b5-4ee7-b7e3-96fef15a2fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.386746 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067a11e-36b5-4ee7-b7e3-96fef15a2fdd admin admin] PATH_INFO: `/v3/projects/1e239d8e525c4631a1eb8f4eede4bed0/users/a26946bb7485409f800e29f66d5e30f2/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.392104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067a11e-36b5-4ee7-b7e3-96fef15a2fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e239d8e525c4631a1eb8f4eede4bed0, user_id=a26946bb7485409f800e29f66d5e30f2, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:23.395873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067a11e-36b5-4ee7-b7e3-96fef15a2fdd admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:23.408841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:23.429213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 925/1824] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:23 2022] PUT /identity/v3/projects/1e239d8e525c4631a1eb8f4eede4bed0/users/a26946bb7485409f800e29f66d5e30f2/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:23.431598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6ba1c-5d0a-4e64-827e-0b12cd789962 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.432361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6ba1c-5d0a-4e64-827e-0b12cd789962 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.432779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6ba1c-5d0a-4e64-827e-0b12cd789962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.471291 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-b1d6ba1c-5d0a-4e64-827e-0b12cd789962 None None] MFA Rules not processed for user `a26946bb7485409f800e29f66d5e30f2`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:23.482665 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b1d6ba1c-5d0a-4e64-827e-0b12cd789962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:23.664394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 900/1825] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:23.677840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067a11e-36b5-4ee7-b7e3-96fef15a2fdd admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:23.680374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7067a11e-36b5-4ee7-b7e3-96fef15a2fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:23.841687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067a11e-36b5-4ee7-b7e3-96fef15a2fdd admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:23.846915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82154b64-87f2-404b-9b36-8c2300612817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:23.847781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82154b64-87f2-404b-9b36-8c2300612817 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.848209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82154b64-87f2-404b-9b36-8c2300612817 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.848581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82154b64-87f2-404b-9b36-8c2300612817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.865097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82154b64-87f2-404b-9b36-8c2300612817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:23.866779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82154b64-87f2-404b-9b36-8c2300612817 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:23.895683 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 926/1826] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:50:24.686991 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6ba1c-5d0a-4e64-827e-0b12cd789962 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:24.794111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6ba1c-5d0a-4e64-827e-0b12cd789962 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:24.801147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:24.802147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.802651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.803280 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] PATH_INFO: `/v3/users/d8e3738cbc714babbeb8742eeff97ec2` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.805465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e3738cbc714babbeb8742eeff97ec2)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:24.808392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:24.871887 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e3738cbc714babbeb8742eeff97ec2'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:24.882009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e3738cbc714babbeb8742eeff97ec2'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:24.882827 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e3738cbc714babbeb8742eeff97ec2'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:24.899171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e3738cbc714babbeb8742eeff97ec2'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:24.905010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:24.921164 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e3738cbc714babbeb8742eeff97ec2'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:24.945117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 901/1827] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:24 2022] DELETE /identity/v3/users/d8e3738cbc714babbeb8742eeff97ec2 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:24.950901 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c0e62-d31f-4d6c-aa74-6ef26b001e63 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.951488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c0e62-d31f-4d6c-aa74-6ef26b001e63 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.951587 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c0e62-d31f-4d6c-aa74-6ef26b001e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.978076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-a22c0e62-d31f-4d6c-aa74-6ef26b001e63 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:24.979740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a22c0e62-d31f-4d6c-aa74-6ef26b001e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:25.056672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 927/1828] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:25.083484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.084925 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:25.181185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dcd43-3697-4628-8865-a2bbb15a55e0 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.187674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e0b94-9957-425d-9cf0-e7ba14eb65d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.188386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e0b94-9957-425d-9cf0-e7ba14eb65d6 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.188707 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e0b94-9957-425d-9cf0-e7ba14eb65d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.188981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e0b94-9957-425d-9cf0-e7ba14eb65d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.208592 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e0b94-9957-425d-9cf0-e7ba14eb65d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.209513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e0b94-9957-425d-9cf0-e7ba14eb65d6 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.224980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 902/1829] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:26.013420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c0e62-d31f-4d6c-aa74-6ef26b001e63 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.015234 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a22c0e62-d31f-4d6c-aa74-6ef26b001e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:26.037987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c0e62-d31f-4d6c-aa74-6ef26b001e63 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.049137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.050540 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.051092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.051548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] PATH_INFO: `/v3/projects/7351b72568704b5795091c260d2e6aca` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.053136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7351b72568704b5795091c260d2e6aca)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.055910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.147319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7351b72568704b5795091c260d2e6aca', 'request_id': 'req-ddaa2d8c-f391-46c9-a788-7517732bc85b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:26.157128 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7351b72568704b5795091c260d2e6aca', 'request_id': 'req-ddaa2d8c-f391-46c9-a788-7517732bc85b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:26.172048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7351b72568704b5795091c260d2e6aca', 'request_id': 'req-ddaa2d8c-f391-46c9-a788-7517732bc85b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:26.191277 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] The token cache is being invalidate because project 7351b72568704b5795091c260d2e6aca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:26.191809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:26.192729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:26.198577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 928/1830] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:26 2022] DELETE /identity/v3/projects/7351b72568704b5795091c260d2e6aca => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:26.201526 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e0b94-9957-425d-9cf0-e7ba14eb65d6 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.216511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e0b94-9957-425d-9cf0-e7ba14eb65d6 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.329152 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b853c7-8d21-4fac-bb18-2c3834352544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.329894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b853c7-8d21-4fac-bb18-2c3834352544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.330182 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b853c7-8d21-4fac-bb18-2c3834352544 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.330459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b853c7-8d21-4fac-bb18-2c3834352544 admin admin] PATH_INFO: `/v3/users/a26946bb7485409f800e29f66d5e30f2` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.334965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b853c7-8d21-4fac-bb18-2c3834352544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a26946bb7485409f800e29f66d5e30f2)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.337137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b853c7-8d21-4fac-bb18-2c3834352544 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.448345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-99b853c7-8d21-4fac-bb18-2c3834352544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a26946bb7485409f800e29f66d5e30f2'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:26.458877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-99b853c7-8d21-4fac-bb18-2c3834352544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a26946bb7485409f800e29f66d5e30f2'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:26.461605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-99b853c7-8d21-4fac-bb18-2c3834352544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a26946bb7485409f800e29f66d5e30f2'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:26.479294 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-99b853c7-8d21-4fac-bb18-2c3834352544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a26946bb7485409f800e29f66d5e30f2'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:26.490756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-99b853c7-8d21-4fac-bb18-2c3834352544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a26946bb7485409f800e29f66d5e30f2'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:26.498553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 903/1831] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:26 2022] DELETE /identity/v3/users/a26946bb7485409f800e29f66d5e30f2 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:26.921655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.923264 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:26.938247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa2d8c-f391-46c9-a788-7517732bc85b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.968975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:27.008735 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff022c-4d96-4564-bba2-9f7faee7c724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.009406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff022c-4d96-4564-bba2-9f7faee7c724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.009686 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff022c-4d96-4564-bba2-9f7faee7c724 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.010791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff022c-4d96-4564-bba2-9f7faee7c724 admin admin] PATH_INFO: `/v3/projects/1e239d8e525c4631a1eb8f4eede4bed0` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.012128 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff022c-4d96-4564-bba2-9f7faee7c724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e239d8e525c4631a1eb8f4eede4bed0)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.014093 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff022c-4d96-4564-bba2-9f7faee7c724 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.066467 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-07ff022c-4d96-4564-bba2-9f7faee7c724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e239d8e525c4631a1eb8f4eede4bed0', 'request_id': 'req-07ff022c-4d96-4564-bba2-9f7faee7c724'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:27.073803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-07ff022c-4d96-4564-bba2-9f7faee7c724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e239d8e525c4631a1eb8f4eede4bed0', 'request_id': 'req-07ff022c-4d96-4564-bba2-9f7faee7c724'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:27.082483 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-07ff022c-4d96-4564-bba2-9f7faee7c724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e239d8e525c4631a1eb8f4eede4bed0', 'request_id': 'req-07ff022c-4d96-4564-bba2-9f7faee7c724'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:27.095344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-07ff022c-4d96-4564-bba2-9f7faee7c724 admin admin] The token cache is being invalidate because project 1e239d8e525c4631a1eb8f4eede4bed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:27.095633 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-07ff022c-4d96-4564-bba2-9f7faee7c724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:27.096326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-07ff022c-4d96-4564-bba2-9f7faee7c724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:27.097924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 929/1832] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:26 2022] DELETE /identity/v3/projects/1e239d8e525c4631a1eb8f4eede4bed0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:27.102766 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458c741-6fbd-44f8-bbd5-bcb13bb0863b None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.103059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458c741-6fbd-44f8-bbd5-bcb13bb0863b None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.103335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458c741-6fbd-44f8-bbd5-bcb13bb0863b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.123403 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-6458c741-6fbd-44f8-bbd5-bcb13bb0863b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:27.124526 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6458c741-6fbd-44f8-bbd5-bcb13bb0863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:27.136043 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:27.192187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 904/1833] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:27.200758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d06d33-3832-44e6-b96a-de07d572a1f4 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.201212 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d06d33-3832-44e6-b96a-de07d572a1f4 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.201372 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d06d33-3832-44e6-b96a-de07d572a1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.221335 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-36d06d33-3832-44e6-b96a-de07d572a1f4 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:27.222693 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-36d06d33-3832-44e6-b96a-de07d572a1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:27.232014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 930/1834] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:27.236470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458c741-6fbd-44f8-bbd5-bcb13bb0863b None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.247506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458c741-6fbd-44f8-bbd5-bcb13bb0863b None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.254706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201d035-7539-43fd-b9e5-64138fea7d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.254706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201d035-7539-43fd-b9e5-64138fea7d3b admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.254706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201d035-7539-43fd-b9e5-64138fea7d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.254706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201d035-7539-43fd-b9e5-64138fea7d3b admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.255376 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201d035-7539-43fd-b9e5-64138fea7d3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:27.255376 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201d035-7539-43fd-b9e5-64138fea7d3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.256327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201d035-7539-43fd-b9e5-64138fea7d3b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.271912 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 905/1835] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:27.276933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d06d33-3832-44e6-b96a-de07d572a1f4 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.286924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d06d33-3832-44e6-b96a-de07d572a1f4 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.292406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afb23c-568c-44b4-a532-a56b856535dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.293285 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afb23c-568c-44b4-a532-a56b856535dd admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.293432 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afb23c-568c-44b4-a532-a56b856535dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.293619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afb23c-568c-44b4-a532-a56b856535dd admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.294278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3afb23c-568c-44b4-a532-a56b856535dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.296212 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3afb23c-568c-44b4-a532-a56b856535dd admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.333099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 931/1836] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:27 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:27.337293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201d035-7539-43fd-b9e5-64138fea7d3b admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.348400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201d035-7539-43fd-b9e5-64138fea7d3b admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.421791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5facfd5b-5f99-40dc-817e-9e36072d38b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.422422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facfd5b-5f99-40dc-817e-9e36072d38b0 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.422742 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facfd5b-5f99-40dc-817e-9e36072d38b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.423006 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5facfd5b-5f99-40dc-817e-9e36072d38b0 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.423672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5facfd5b-5f99-40dc-817e-9e36072d38b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.425599 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5facfd5b-5f99-40dc-817e-9e36072d38b0 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.469371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 906/1837] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:27 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:27.473235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afb23c-568c-44b4-a532-a56b856535dd admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.485773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afb23c-568c-44b4-a532-a56b856535dd admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.491271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e14da-8360-44e2-a5a2-8f18774b1bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.491899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e14da-8360-44e2-a5a2-8f18774b1bfe admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.492165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e14da-8360-44e2-a5a2-8f18774b1bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.492444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e14da-8360-44e2-a5a2-8f18774b1bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.493196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e14da-8360-44e2-a5a2-8f18774b1bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.495560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e14da-8360-44e2-a5a2-8f18774b1bfe admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.512144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5facfd5b-5f99-40dc-817e-9e36072d38b0 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.512940 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 932/1838] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:27.539642 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5facfd5b-5f99-40dc-817e-9e36072d38b0 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.548692 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115495fa-677a-4c23-9064-b86a76563dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.549788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115495fa-677a-4c23-9064-b86a76563dcf admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.550470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115495fa-677a-4c23-9064-b86a76563dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.550974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115495fa-677a-4c23-9064-b86a76563dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.551894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115495fa-677a-4c23-9064-b86a76563dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.555924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115495fa-677a-4c23-9064-b86a76563dcf admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.592829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 907/1839] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:50:27.595138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e14da-8360-44e2-a5a2-8f18774b1bfe admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.628422 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e14da-8360-44e2-a5a2-8f18774b1bfe admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.636613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98291da-680f-40eb-a341-4014078fbdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.637839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98291da-680f-40eb-a341-4014078fbdfc admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.638903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98291da-680f-40eb-a341-4014078fbdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.638903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98291da-680f-40eb-a341-4014078fbdfc admin admin] PATH_INFO: `/v3/projects/6d86ae2978c34fcaacdaf63eca3706f8/users/4373be2e40c940a89a3c3c6f4f306b14/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.664769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98291da-680f-40eb-a341-4014078fbdfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d86ae2978c34fcaacdaf63eca3706f8, user_id=4373be2e40c940a89a3c3c6f4f306b14, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.669890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98291da-680f-40eb-a341-4014078fbdfc admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.690937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 933/1840] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:27 2022] PUT /identity/v3/projects/6d86ae2978c34fcaacdaf63eca3706f8/users/4373be2e40c940a89a3c3c6f4f306b14/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:27.696122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096d379-0eb6-4d71-bc38-ce77519825ac None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.696544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096d379-0eb6-4d71-bc38-ce77519825ac None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.696918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096d379-0eb6-4d71-bc38-ce77519825ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.747717 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-d096d379-0eb6-4d71-bc38-ce77519825ac None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:27.749782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d096d379-0eb6-4d71-bc38-ce77519825ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:27.826853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 908/1841] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:27.875692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98291da-680f-40eb-a341-4014078fbdfc admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.877691 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a98291da-680f-40eb-a341-4014078fbdfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:27.984510 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98291da-680f-40eb-a341-4014078fbdfc admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.988811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2e7e2-9117-47d9-abd4-3eb60cf73281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.989882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2e7e2-9117-47d9-abd4-3eb60cf73281 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.991260 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2e7e2-9117-47d9-abd4-3eb60cf73281 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.991657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2e7e2-9117-47d9-abd4-3eb60cf73281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.002616 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:28.006744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2e7e2-9117-47d9-abd4-3eb60cf73281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:28.008544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2e7e2-9117-47d9-abd4-3eb60cf73281 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:28.024627 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 934/1842] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:29.151938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dc235-aad3-45b6-a915-4817644d43de None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.151938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dc235-aad3-45b6-a915-4817644d43de None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.152587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dc235-aad3-45b6-a915-4817644d43de None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.155027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 909/1843] 10.222.0.86 () {54 vars in 921 bytes} [Fri Aug 5 19:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:29.202420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6777b1-9814-4288-a6a5-8908c3569dcd None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.203943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6777b1-9814-4288-a6a5-8908c3569dcd None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.204579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6777b1-9814-4288-a6a5-8908c3569dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.236974 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-fd6777b1-9814-4288-a6a5-8908c3569dcd None None] MFA Rules not processed for user `ab8f9d9cf4ca4e38ae6eb67e949b40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:29.239190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fd6777b1-9814-4288-a6a5-8908c3569dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:29.345155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 935/1844] 10.222.0.86 () {58 vars in 1018 bytes} [Fri Aug 5 19:50:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:29.424093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dc235-aad3-45b6-a915-4817644d43de None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.426137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-0d3dc235-aad3-45b6-a915-4817644d43de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:29.462924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dc235-aad3-45b6-a915-4817644d43de None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:29.486963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb3cb1-32ad-4a2b-a690-e00cc033e414 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:29.503706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb3cb1-32ad-4a2b-a690-e00cc033e414 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.511327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb3cb1-32ad-4a2b-a690-e00cc033e414 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.511651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb3cb1-32ad-4a2b-a690-e00cc033e414 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.526332 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb3cb1-32ad-4a2b-a690-e00cc033e414 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:29.528362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb3cb1-32ad-4a2b-a690-e00cc033e414 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:29.551693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 910/1845] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:30.395455 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e100f5a-09f2-4bd5-833d-e62b57295e67 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.396451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e100f5a-09f2-4bd5-833d-e62b57295e67 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.396451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e100f5a-09f2-4bd5-833d-e62b57295e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.429726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-2e100f5a-09f2-4bd5-833d-e62b57295e67 None None] MFA Rules not processed for user `4373be2e40c940a89a3c3c6f4f306b14`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:30.432483 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2e100f5a-09f2-4bd5-833d-e62b57295e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:30.538129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 936/1846] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:30.543581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb3cb1-32ad-4a2b-a690-e00cc033e414 service nova] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.562964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb3cb1-32ad-4a2b-a690-e00cc033e414 service nova] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.568675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3f29b-53c2-4133-9175-f1c6514c6629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.569407 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3f29b-53c2-4133-9175-f1c6514c6629 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.569686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3f29b-53c2-4133-9175-f1c6514c6629 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.569951 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3f29b-53c2-4133-9175-f1c6514c6629 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.570944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3f29b-53c2-4133-9175-f1c6514c6629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.575444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3f29b-53c2-4133-9175-f1c6514c6629 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.584927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.623679 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 911/1847] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:30 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:30.627410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e100f5a-09f2-4bd5-833d-e62b57295e67 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.640435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e100f5a-09f2-4bd5-833d-e62b57295e67 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.718147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b0d6f-1749-4c57-b4ae-80a374d43f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.719111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b0d6f-1749-4c57-b4ae-80a374d43f85 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.719553 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b0d6f-1749-4c57-b4ae-80a374d43f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.719987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b0d6f-1749-4c57-b4ae-80a374d43f85 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.720811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b0d6f-1749-4c57-b4ae-80a374d43f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.723317 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b0d6f-1749-4c57-b4ae-80a374d43f85 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.758254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 937/1848] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:30 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:30.762810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3f29b-53c2-4133-9175-f1c6514c6629 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.774508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3f29b-53c2-4133-9175-f1c6514c6629 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.780985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39802961-a8ea-48d6-ad6e-ca34fd4d8472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.781937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39802961-a8ea-48d6-ad6e-ca34fd4d8472 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.782436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39802961-a8ea-48d6-ad6e-ca34fd4d8472 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.783029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39802961-a8ea-48d6-ad6e-ca34fd4d8472 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.783978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39802961-a8ea-48d6-ad6e-ca34fd4d8472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.786011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39802961-a8ea-48d6-ad6e-ca34fd4d8472 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.803195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 912/1849] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:30.804145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b0d6f-1749-4c57-b4ae-80a374d43f85 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.816538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b0d6f-1749-4c57-b4ae-80a374d43f85 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.822183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b87380-735b-42f8-851b-9a6bad96b280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.823851 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b87380-735b-42f8-851b-9a6bad96b280 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.824469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b87380-735b-42f8-851b-9a6bad96b280 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.826172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b87380-735b-42f8-851b-9a6bad96b280 admin admin] PATH_INFO: `/v3/projects/2f5fcc7816fa47fc90c10cce190e3017/users/8902773db06844279438b875ab727425/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.829537 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.846925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b87380-735b-42f8-851b-9a6bad96b280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f5fcc7816fa47fc90c10cce190e3017, user_id=8902773db06844279438b875ab727425, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.850459 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b87380-735b-42f8-851b-9a6bad96b280 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.869517 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 938/1850] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:30 2022] PUT /identity/v3/projects/2f5fcc7816fa47fc90c10cce190e3017/users/8902773db06844279438b875ab727425/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:30.875453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39802961-a8ea-48d6-ad6e-ca34fd4d8472 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.898002 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39802961-a8ea-48d6-ad6e-ca34fd4d8472 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.019653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b46449-968c-4e3d-8627-eb8341f28d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.019962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b46449-968c-4e3d-8627-eb8341f28d2a admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.022573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b46449-968c-4e3d-8627-eb8341f28d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.023653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b46449-968c-4e3d-8627-eb8341f28d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.024399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b46449-968c-4e3d-8627-eb8341f28d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:31.027323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b46449-968c-4e3d-8627-eb8341f28d2a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:31.039848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 913/1851] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:31.045095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b87380-735b-42f8-851b-9a6bad96b280 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.063801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b87380-735b-42f8-851b-9a6bad96b280 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.076294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae981e2-09ae-40dd-9a9c-aacb50a623c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.077456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae981e2-09ae-40dd-9a9c-aacb50a623c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.078085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae981e2-09ae-40dd-9a9c-aacb50a623c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.078832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae981e2-09ae-40dd-9a9c-aacb50a623c6 admin admin] PATH_INFO: `/v3/projects/2f5fcc7816fa47fc90c10cce190e3017/users/8902773db06844279438b875ab727425/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.081660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae981e2-09ae-40dd-9a9c-aacb50a623c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f5fcc7816fa47fc90c10cce190e3017, user_id=8902773db06844279438b875ab727425, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:31.084934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae981e2-09ae-40dd-9a9c-aacb50a623c6 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:31.108134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 939/1852] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:31 2022] PUT /identity/v3/projects/2f5fcc7816fa47fc90c10cce190e3017/users/8902773db06844279438b875ab727425/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:32.552518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca43065-a8f6-4e63-8b9a-363f75b83f91 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.553028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca43065-a8f6-4e63-8b9a-363f75b83f91 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.553137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca43065-a8f6-4e63-8b9a-363f75b83f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.583569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-dca43065-a8f6-4e63-8b9a-363f75b83f91 None None] MFA Rules not processed for user `8902773db06844279438b875ab727425`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:32.584806 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-dca43065-a8f6-4e63-8b9a-363f75b83f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:32.666778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 914/1853] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:32.677265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae981e2-09ae-40dd-9a9c-aacb50a623c6 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:32.683950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8ae981e2-09ae-40dd-9a9c-aacb50a623c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:32.752381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae981e2-09ae-40dd-9a9c-aacb50a623c6 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:32.764221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e7252-61e6-4af1-9dca-5ff16111639a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:32.764867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e7252-61e6-4af1-9dca-5ff16111639a service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:32.765145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e7252-61e6-4af1-9dca-5ff16111639a service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:32.765446 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e7252-61e6-4af1-9dca-5ff16111639a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:32.831291 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e7252-61e6-4af1-9dca-5ff16111639a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:32.832736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e7252-61e6-4af1-9dca-5ff16111639a service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:32.843173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 940/1854] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:35.400374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca43065-a8f6-4e63-8b9a-363f75b83f91 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:35.412235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca43065-a8f6-4e63-8b9a-363f75b83f91 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:35.428045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5edafb-491d-41c8-9036-ca2cb2431204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:35.430377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5edafb-491d-41c8-9036-ca2cb2431204 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:35.431495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5edafb-491d-41c8-9036-ca2cb2431204 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:35.433731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5edafb-491d-41c8-9036-ca2cb2431204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:35.452097 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5edafb-491d-41c8-9036-ca2cb2431204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:35.454978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5edafb-491d-41c8-9036-ca2cb2431204 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:35.487263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 915/1855] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:36.329319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e7252-61e6-4af1-9dca-5ff16111639a service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:36.331060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9d0e7252-61e6-4af1-9dca-5ff16111639a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:36.409132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e7252-61e6-4af1-9dca-5ff16111639a service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:36.413779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:36.414431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.414742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.415016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] PATH_INFO: `/v3/users/46c447b35cd14422baaf480f22dc1ba3` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.416400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] RBAC: Authorizing `identity:delete_user(user_id=46c447b35cd14422baaf480f22dc1ba3)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:36.418435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:36.451615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:36.473041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46c447b35cd14422baaf480f22dc1ba3'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:36.487307 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46c447b35cd14422baaf480f22dc1ba3'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:36.496773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46c447b35cd14422baaf480f22dc1ba3'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:36.497416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46c447b35cd14422baaf480f22dc1ba3'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:36.498011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46c447b35cd14422baaf480f22dc1ba3'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:36.514443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 941/1856] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:36 2022] DELETE /identity/v3/users/46c447b35cd14422baaf480f22dc1ba3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:36.519091 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f70f88-e075-4ffe-a391-7cb27057f032 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.519574 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f70f88-e075-4ffe-a391-7cb27057f032 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.519676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f70f88-e075-4ffe-a391-7cb27057f032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.545943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c1f70f88-e075-4ffe-a391-7cb27057f032 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:36.548456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c1f70f88-e075-4ffe-a391-7cb27057f032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:36.632363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:36.662613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 916/1857] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:36.678453 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:36.680406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:36.749655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0bb1f-9be7-4c0a-9d20-a9668dd0822e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:36.753242 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94229d06-38c9-4b93-93bf-47653c05a847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:36.753868 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94229d06-38c9-4b93-93bf-47653c05a847 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.754144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94229d06-38c9-4b93-93bf-47653c05a847 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.754780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94229d06-38c9-4b93-93bf-47653c05a847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.767032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94229d06-38c9-4b93-93bf-47653c05a847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:36.768498 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94229d06-38c9-4b93-93bf-47653c05a847 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:36.778870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 942/1858] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:37.218320 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f70f88-e075-4ffe-a391-7cb27057f032 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.219782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c1f70f88-e075-4ffe-a391-7cb27057f032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:37.235901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f70f88-e075-4ffe-a391-7cb27057f032 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.254974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.255644 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.255916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.256183 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] PATH_INFO: `/v3/projects/0b2146a5f18b4224b571bebf542704dd` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.263971 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2146a5f18b4224b571bebf542704dd)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:37.266146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:37.454001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2146a5f18b4224b571bebf542704dd', 'request_id': 'req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:37.465897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2146a5f18b4224b571bebf542704dd', 'request_id': 'req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:37.490912 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2146a5f18b4224b571bebf542704dd', 'request_id': 'req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:37.506800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] The token cache is being invalidate because project 0b2146a5f18b4224b571bebf542704dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:37.507040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:37.507935 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:37.510767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 917/1859] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:37 2022] DELETE /identity/v3/projects/0b2146a5f18b4224b571bebf542704dd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:37.514264 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94229d06-38c9-4b93-93bf-47653c05a847 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.530144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94229d06-38c9-4b93-93bf-47653c05a847 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.604851 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a95da-1c6c-4a4f-b4e7-479d6b9de942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.605074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a95da-1c6c-4a4f-b4e7-479d6b9de942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.605378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a95da-1c6c-4a4f-b4e7-479d6b9de942 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.605723 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a95da-1c6c-4a4f-b4e7-479d6b9de942 admin admin] PATH_INFO: `/v3/users/d24f9afb647142bfa6b243037d0ce20c` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.607339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a95da-1c6c-4a4f-b4e7-479d6b9de942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d24f9afb647142bfa6b243037d0ce20c)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:37.609656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a95da-1c6c-4a4f-b4e7-479d6b9de942 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:37.664996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-613a95da-1c6c-4a4f-b4e7-479d6b9de942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd24f9afb647142bfa6b243037d0ce20c'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:37.666487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:37.679600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-613a95da-1c6c-4a4f-b4e7-479d6b9de942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd24f9afb647142bfa6b243037d0ce20c'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:37.687802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-613a95da-1c6c-4a4f-b4e7-479d6b9de942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd24f9afb647142bfa6b243037d0ce20c'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:37.688408 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-613a95da-1c6c-4a4f-b4e7-479d6b9de942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd24f9afb647142bfa6b243037d0ce20c'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:37.689026 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-613a95da-1c6c-4a4f-b4e7-479d6b9de942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd24f9afb647142bfa6b243037d0ce20c'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:37.702238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 943/1860] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:37 2022] DELETE /identity/v3/users/d24f9afb647142bfa6b243037d0ce20c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:38.050078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.051760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:38.067543 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b83b69-8d2b-4f3b-8494-48c230f4cbb9 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.128061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.128757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.129177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.129349 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185 admin admin] PATH_INFO: `/v3/projects/63eaf9e424af4739a88ff0f3a12ef464` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.130637 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63eaf9e424af4739a88ff0f3a12ef464)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.132612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.182689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63eaf9e424af4739a88ff0f3a12ef464', 'request_id': 'req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:38.190045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63eaf9e424af4739a88ff0f3a12ef464', 'request_id': 'req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:38.200165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63eaf9e424af4739a88ff0f3a12ef464', 'request_id': 'req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:38.212018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:38.217663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185 admin admin] The token cache is being invalidate because project 63eaf9e424af4739a88ff0f3a12ef464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:38.218324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:38.219199 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-0cc8f119-c5cf-4a0a-843e-a2bf446c4185 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:38.220738 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 918/1861] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:38 2022] DELETE /identity/v3/projects/63eaf9e424af4739a88ff0f3a12ef464 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:38.226764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d8840-5a23-4ca4-b620-36801ca7ddb4 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.227236 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d8840-5a23-4ca4-b620-36801ca7ddb4 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.230349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d8840-5a23-4ca4-b620-36801ca7ddb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.261308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-d05d8840-5a23-4ca4-b620-36801ca7ddb4 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:38.263739 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d05d8840-5a23-4ca4-b620-36801ca7ddb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:38.345082 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 944/1862] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:38.353874 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67073137-dfe1-43b2-918c-aed732bed9b3 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.354185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67073137-dfe1-43b2-918c-aed732bed9b3 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.354428 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67073137-dfe1-43b2-918c-aed732bed9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.376699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-67073137-dfe1-43b2-918c-aed732bed9b3 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:38.378166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-67073137-dfe1-43b2-918c-aed732bed9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:38.387417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 919/1863] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:38.391875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d8840-5a23-4ca4-b620-36801ca7ddb4 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.402777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d8840-5a23-4ca4-b620-36801ca7ddb4 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.407475 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca714e93-145d-4699-a698-2f99bd87d06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.408093 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca714e93-145d-4699-a698-2f99bd87d06d admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.408368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca714e93-145d-4699-a698-2f99bd87d06d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.408635 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca714e93-145d-4699-a698-2f99bd87d06d admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.409329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca714e93-145d-4699-a698-2f99bd87d06d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:38.409628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca714e93-145d-4699-a698-2f99bd87d06d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.411250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca714e93-145d-4699-a698-2f99bd87d06d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.424886 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 945/1864] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:38.429041 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67073137-dfe1-43b2-918c-aed732bed9b3 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.438975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67073137-dfe1-43b2-918c-aed732bed9b3 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.443699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c6685-2187-402c-b971-eb9d77eeb332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.444247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c6685-2187-402c-b971-eb9d77eeb332 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.446053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c6685-2187-402c-b971-eb9d77eeb332 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.446346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c6685-2187-402c-b971-eb9d77eeb332 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.447783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c6685-2187-402c-b971-eb9d77eeb332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.450535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c6685-2187-402c-b971-eb9d77eeb332 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.491938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 920/1865] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:38 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:38.496640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca714e93-145d-4699-a698-2f99bd87d06d admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.511755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca714e93-145d-4699-a698-2f99bd87d06d admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.651464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25dab7-7b3a-43e8-ad4e-c6274a7c023d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.652223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25dab7-7b3a-43e8-ad4e-c6274a7c023d admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.653302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25dab7-7b3a-43e8-ad4e-c6274a7c023d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.653696 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25dab7-7b3a-43e8-ad4e-c6274a7c023d admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.654462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa25dab7-7b3a-43e8-ad4e-c6274a7c023d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.657277 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa25dab7-7b3a-43e8-ad4e-c6274a7c023d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.691472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 946/1866] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:38 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:38.697774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c6685-2187-402c-b971-eb9d77eeb332 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.711825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c6685-2187-402c-b971-eb9d77eeb332 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.723071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fbc4b-c486-4977-a09a-7087488e558c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.724503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fbc4b-c486-4977-a09a-7087488e558c admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.724931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fbc4b-c486-4977-a09a-7087488e558c admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.725388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fbc4b-c486-4977-a09a-7087488e558c admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.727009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fbc4b-c486-4977-a09a-7087488e558c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.729256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fbc4b-c486-4977-a09a-7087488e558c admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.750962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 921/1867] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:38.755774 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25dab7-7b3a-43e8-ad4e-c6274a7c023d admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.777817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25dab7-7b3a-43e8-ad4e-c6274a7c023d admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.788319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96b6e1-c452-4f74-b8ce-c225ebb8f502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.788996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96b6e1-c452-4f74-b8ce-c225ebb8f502 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.789232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96b6e1-c452-4f74-b8ce-c225ebb8f502 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.789501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96b6e1-c452-4f74-b8ce-c225ebb8f502 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.790325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96b6e1-c452-4f74-b8ce-c225ebb8f502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.792278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96b6e1-c452-4f74-b8ce-c225ebb8f502 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.808186 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 947/1868] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:38.809044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fbc4b-c486-4977-a09a-7087488e558c admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.819774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fbc4b-c486-4977-a09a-7087488e558c admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.826939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4103efd-335f-4669-85cb-2c89ab4b7fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.827743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4103efd-335f-4669-85cb-2c89ab4b7fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.828473 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4103efd-335f-4669-85cb-2c89ab4b7fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.828589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4103efd-335f-4669-85cb-2c89ab4b7fc1 admin admin] PATH_INFO: `/v3/projects/6a91ceb60d9c401db2acc6d20aa051d1/users/aca2796cc22c42679f5de6f4339ccccf/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.846082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4103efd-335f-4669-85cb-2c89ab4b7fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a91ceb60d9c401db2acc6d20aa051d1, user_id=aca2796cc22c42679f5de6f4339ccccf, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.847454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4103efd-335f-4669-85cb-2c89ab4b7fc1 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.865663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 922/1869] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:38 2022] PUT /identity/v3/projects/6a91ceb60d9c401db2acc6d20aa051d1/users/aca2796cc22c42679f5de6f4339ccccf/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:38.876320 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f4ec0-88da-4af3-8862-85c194971d40 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.876606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f4ec0-88da-4af3-8862-85c194971d40 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.876875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f4ec0-88da-4af3-8862-85c194971d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.908750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-629f4ec0-88da-4af3-8862-85c194971d40 None None] MFA Rules not processed for user `aca2796cc22c42679f5de6f4339ccccf`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:38.910648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-629f4ec0-88da-4af3-8862-85c194971d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:38.933880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:39.024052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 948/1870] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:39.029046 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4103efd-335f-4669-85cb-2c89ab4b7fc1 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.091506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0d184-e2a1-42f4-8450-477ea321f652 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.092641 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0d184-e2a1-42f4-8450-477ea321f652 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.093111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0d184-e2a1-42f4-8450-477ea321f652 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.097860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 949/1871] 10.222.0.86 () {54 vars in 921 bytes} [Fri Aug 5 19:50:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:39.123003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884fe1-8934-4c72-aa06-91cc0d70c96d None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.125459 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884fe1-8934-4c72-aa06-91cc0d70c96d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.126170 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884fe1-8934-4c72-aa06-91cc0d70c96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.149573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4103efd-335f-4669-85cb-2c89ab4b7fc1 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.159192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97eebe-adcb-4853-a66c-434c3996aff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.160221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97eebe-adcb-4853-a66c-434c3996aff9 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.160754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97eebe-adcb-4853-a66c-434c3996aff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.161413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97eebe-adcb-4853-a66c-434c3996aff9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.162564 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97eebe-adcb-4853-a66c-434c3996aff9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.167580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97eebe-adcb-4853-a66c-434c3996aff9 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.201369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-d0884fe1-8934-4c72-aa06-91cc0d70c96d None None] MFA Rules not processed for user `ab8f9d9cf4ca4e38ae6eb67e949b40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:39.203330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d0884fe1-8934-4c72-aa06-91cc0d70c96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:39.224148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 923/1872] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:39 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:39.225635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97eebe-adcb-4853-a66c-434c3996aff9 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.356160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97eebe-adcb-4853-a66c-434c3996aff9 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.361261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 950/1873] 10.222.0.86 () {58 vars in 1018 bytes} [Fri Aug 5 19:50:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:39.363278 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada2759-8b12-49c5-87dd-e06f3f78f93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.364064 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada2759-8b12-49c5-87dd-e06f3f78f93b admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.364522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada2759-8b12-49c5-87dd-e06f3f78f93b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.364804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada2759-8b12-49c5-87dd-e06f3f78f93b admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.365547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada2759-8b12-49c5-87dd-e06f3f78f93b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.367708 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada2759-8b12-49c5-87dd-e06f3f78f93b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.384016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0884fe1-8934-4c72-aa06-91cc0d70c96d None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.387982 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d0884fe1-8934-4c72-aa06-91cc0d70c96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:39.409697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 924/1874] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:39 2022] POST /identity/v3/users => generated 300 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:39.418486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada2759-8b12-49c5-87dd-e06f3f78f93b admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.448938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0884fe1-8934-4c72-aa06-91cc0d70c96d None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.455794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada2759-8b12-49c5-87dd-e06f3f78f93b admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.472746 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156cd0f-4f19-43f5-8063-fa54ec17be94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.473658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156cd0f-4f19-43f5-8063-fa54ec17be94 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.475364 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156cd0f-4f19-43f5-8063-fa54ec17be94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.476031 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156cd0f-4f19-43f5-8063-fa54ec17be94 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.477388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156cd0f-4f19-43f5-8063-fa54ec17be94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.481779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156cd0f-4f19-43f5-8063-fa54ec17be94 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.490061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:39.506713 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 925/1875] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:39.515099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156cd0f-4f19-43f5-8063-fa54ec17be94 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.530303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156cd0f-4f19-43f5-8063-fa54ec17be94 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.536485 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069a783-d696-45bb-bc73-22ead48b5eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.537218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069a783-d696-45bb-bc73-22ead48b5eac admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.537503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069a783-d696-45bb-bc73-22ead48b5eac admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.537781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069a783-d696-45bb-bc73-22ead48b5eac admin admin] PATH_INFO: `/v3/projects/60d8cca69be34325a8e6f0abd59a1ca7/users/90c53016a4874f49a15b7370326ae4a0/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.571162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069a783-d696-45bb-bc73-22ead48b5eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d8cca69be34325a8e6f0abd59a1ca7, user_id=90c53016a4874f49a15b7370326ae4a0, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.575730 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069a783-d696-45bb-bc73-22ead48b5eac admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.613486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 926/1876] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:39 2022] PUT /identity/v3/projects/60d8cca69be34325a8e6f0abd59a1ca7/users/90c53016a4874f49a15b7370326ae4a0/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:39.617339 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069a783-d696-45bb-bc73-22ead48b5eac admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.621321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c407c-0e62-4522-b1c8-0d6fd87b120a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.621962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c407c-0e62-4522-b1c8-0d6fd87b120a service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.622239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c407c-0e62-4522-b1c8-0d6fd87b120a service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.623057 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c407c-0e62-4522-b1c8-0d6fd87b120a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.632822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c407c-0e62-4522-b1c8-0d6fd87b120a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.635101 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c407c-0e62-4522-b1c8-0d6fd87b120a service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.662401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 951/1877] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:50:39.761781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069a783-d696-45bb-bc73-22ead48b5eac admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.782241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ea0a3-6027-48e6-b017-1c89c276ee65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.783011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ea0a3-6027-48e6-b017-1c89c276ee65 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.783394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ea0a3-6027-48e6-b017-1c89c276ee65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.783775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ea0a3-6027-48e6-b017-1c89c276ee65 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.784734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ea0a3-6027-48e6-b017-1c89c276ee65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.786569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ea0a3-6027-48e6-b017-1c89c276ee65 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.813265 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 927/1878] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:39.816756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c407c-0e62-4522-b1c8-0d6fd87b120a service nova] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.833879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c407c-0e62-4522-b1c8-0d6fd87b120a service nova] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.845048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9464f8-9bb8-4871-8aa5-6d7e27e77215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.846069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9464f8-9bb8-4871-8aa5-6d7e27e77215 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.846674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9464f8-9bb8-4871-8aa5-6d7e27e77215 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.847114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9464f8-9bb8-4871-8aa5-6d7e27e77215 admin admin] PATH_INFO: `/v3/projects/60d8cca69be34325a8e6f0abd59a1ca7/users/90c53016a4874f49a15b7370326ae4a0/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.850309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9464f8-9bb8-4871-8aa5-6d7e27e77215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d8cca69be34325a8e6f0abd59a1ca7, user_id=90c53016a4874f49a15b7370326ae4a0, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.854357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9464f8-9bb8-4871-8aa5-6d7e27e77215 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.885099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 952/1879] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:39 2022] PUT /identity/v3/projects/60d8cca69be34325a8e6f0abd59a1ca7/users/90c53016a4874f49a15b7370326ae4a0/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:39.886843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8306c-3cce-4ba8-96e9-9e030bbe2000 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.887313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8306c-3cce-4ba8-96e9-9e030bbe2000 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.887586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8306c-3cce-4ba8-96e9-9e030bbe2000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.952311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c6b8306c-3cce-4ba8-96e9-9e030bbe2000 None None] MFA Rules not processed for user `90c53016a4874f49a15b7370326ae4a0`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:39.954220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c6b8306c-3cce-4ba8-96e9-9e030bbe2000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:40.122758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 928/1880] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:40.136505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9464f8-9bb8-4871-8aa5-6d7e27e77215 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:40.138308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ea9464f8-9bb8-4871-8aa5-6d7e27e77215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:40.186280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:40.231865 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9464f8-9bb8-4871-8aa5-6d7e27e77215 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:40.237078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c230d475-4091-4a13-b473-f7f99b50752e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:40.238447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d475-4091-4a13-b473-f7f99b50752e service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:40.238768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d475-4091-4a13-b473-f7f99b50752e service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:40.239051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230d475-4091-4a13-b473-f7f99b50752e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:40.261577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230d475-4091-4a13-b473-f7f99b50752e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:40.263517 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c230d475-4091-4a13-b473-f7f99b50752e service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:40.303113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 953/1881] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:40.376790 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8306c-3cce-4ba8-96e9-9e030bbe2000 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:40.486990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:40.490001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8306c-3cce-4ba8-96e9-9e030bbe2000 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:40.497232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:40.498230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:40.498592 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:40.498904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] PATH_INFO: `/v3/users/aca2796cc22c42679f5de6f4339ccccf` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:40.500696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca2796cc22c42679f5de6f4339ccccf)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:40.502619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:40.552850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca2796cc22c42679f5de6f4339ccccf'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:40.560500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca2796cc22c42679f5de6f4339ccccf'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:40.561202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca2796cc22c42679f5de6f4339ccccf'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:40.573496 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca2796cc22c42679f5de6f4339ccccf'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:40.582135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca2796cc22c42679f5de6f4339ccccf'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:40.589295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 929/1882] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:40 2022] DELETE /identity/v3/users/aca2796cc22c42679f5de6f4339ccccf => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:40.593889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd130d0a-29df-450c-adb3-2cb92f130af4 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:40.594199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd130d0a-29df-450c-adb3-2cb92f130af4 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:40.594471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd130d0a-29df-450c-adb3-2cb92f130af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:40.613781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-dd130d0a-29df-450c-adb3-2cb92f130af4 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:40.615015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-dd130d0a-29df-450c-adb3-2cb92f130af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:40.674231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 954/1883] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:40.684503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:40.685967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:40.769576 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127ea09-eb55-410e-a32a-504f0357d5be admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:40.775753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965fed7-1f0e-45c5-8d94-bcf4ca68ba3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:40.777330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965fed7-1f0e-45c5-8d94-bcf4ca68ba3f service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:40.778767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965fed7-1f0e-45c5-8d94-bcf4ca68ba3f service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:40.779722 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965fed7-1f0e-45c5-8d94-bcf4ca68ba3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:40.810208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965fed7-1f0e-45c5-8d94-bcf4ca68ba3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:40.812081 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965fed7-1f0e-45c5-8d94-bcf4ca68ba3f service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:40.827255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 930/1884] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:41.119402 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd130d0a-29df-450c-adb3-2cb92f130af4 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:41.121279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-dd130d0a-29df-450c-adb3-2cb92f130af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:41.140054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd130d0a-29df-450c-adb3-2cb92f130af4 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:41.153677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:41.154757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.155283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.155698 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] PATH_INFO: `/v3/users/399994deb42e46e399e82d3db4fa7876` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.157109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=399994deb42e46e399e82d3db4fa7876)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:41.163719 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:41.241403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '399994deb42e46e399e82d3db4fa7876'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:41.263125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '399994deb42e46e399e82d3db4fa7876'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:41.278871 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '399994deb42e46e399e82d3db4fa7876'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:41.279777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '399994deb42e46e399e82d3db4fa7876'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:41.280906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '399994deb42e46e399e82d3db4fa7876'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:41.303781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 955/1885] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:41 2022] DELETE /identity/v3/users/399994deb42e46e399e82d3db4fa7876 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:41.309650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfa396-17a6-4c4e-9fb3-0efe1877f645 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.309932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfa396-17a6-4c4e-9fb3-0efe1877f645 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.310202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfa396-17a6-4c4e-9fb3-0efe1877f645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.336660 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-f6bfa396-17a6-4c4e-9fb3-0efe1877f645 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:41.338001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f6bfa396-17a6-4c4e-9fb3-0efe1877f645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:41.443459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 931/1886] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:41.496803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:41.499136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:41.581564 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:41.602450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b91612-6b8c-494d-ac3b-af14f95349bc admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:41.607773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03ef1b-1b42-49b5-82a4-38cbc01fc5bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:41.608996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03ef1b-1b42-49b5-82a4-38cbc01fc5bd service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.609449 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03ef1b-1b42-49b5-82a4-38cbc01fc5bd service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.610376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03ef1b-1b42-49b5-82a4-38cbc01fc5bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.621614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03ef1b-1b42-49b5-82a4-38cbc01fc5bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:41.623749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03ef1b-1b42-49b5-82a4-38cbc01fc5bd service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:41.651375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 956/1887] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:42.038199 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfa396-17a6-4c4e-9fb3-0efe1877f645 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.039646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f6bfa396-17a6-4c4e-9fb3-0efe1877f645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:42.065742 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfa396-17a6-4c4e-9fb3-0efe1877f645 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.074773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.075447 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.075775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.076060 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] PATH_INFO: `/v3/projects/6a91ceb60d9c401db2acc6d20aa051d1` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.077374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a91ceb60d9c401db2acc6d20aa051d1)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.079682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.093975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:42.191790 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a91ceb60d9c401db2acc6d20aa051d1', 'request_id': 'req-cdbf634e-a7a9-4458-a090-617f49d915d4'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.209453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a91ceb60d9c401db2acc6d20aa051d1', 'request_id': 'req-cdbf634e-a7a9-4458-a090-617f49d915d4'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.223335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a91ceb60d9c401db2acc6d20aa051d1', 'request_id': 'req-cdbf634e-a7a9-4458-a090-617f49d915d4'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.246945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] The token cache is being invalidate because project 6a91ceb60d9c401db2acc6d20aa051d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:42.248623 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.252777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.260571 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 932/1888] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:42 2022] DELETE /identity/v3/projects/6a91ceb60d9c401db2acc6d20aa051d1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:42.261796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03ef1b-1b42-49b5-82a4-38cbc01fc5bd service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.285772 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03ef1b-1b42-49b5-82a4-38cbc01fc5bd service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.389366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.390027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.390298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.390565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] PATH_INFO: `/v3/users/90c53016a4874f49a15b7370326ae4a0` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.391792 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c53016a4874f49a15b7370326ae4a0)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.393719 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.457821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c53016a4874f49a15b7370326ae4a0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.474891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c53016a4874f49a15b7370326ae4a0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.489084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c53016a4874f49a15b7370326ae4a0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.489879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c53016a4874f49a15b7370326ae4a0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.490744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c53016a4874f49a15b7370326ae4a0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.514495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 957/1889] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:42 2022] DELETE /identity/v3/users/90c53016a4874f49a15b7370326ae4a0 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:42.613022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.614580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:42.631884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf634e-a7a9-4458-a090-617f49d915d4 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.731743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b68b6b-b08e-4625-917a-8626b7cae6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.733130 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b68b6b-b08e-4625-917a-8626b7cae6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.733570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b68b6b-b08e-4625-917a-8626b7cae6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.734182 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b68b6b-b08e-4625-917a-8626b7cae6bd admin admin] PATH_INFO: `/v3/projects/669187f32db34f59ac18dc23a4cbd8a3` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.735756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b68b6b-b08e-4625-917a-8626b7cae6bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=669187f32db34f59ac18dc23a4cbd8a3)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.738010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b68b6b-b08e-4625-917a-8626b7cae6bd admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.819976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-04b68b6b-b08e-4625-917a-8626b7cae6bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '669187f32db34f59ac18dc23a4cbd8a3', 'request_id': 'req-04b68b6b-b08e-4625-917a-8626b7cae6bd'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.829593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-04b68b6b-b08e-4625-917a-8626b7cae6bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '669187f32db34f59ac18dc23a4cbd8a3', 'request_id': 'req-04b68b6b-b08e-4625-917a-8626b7cae6bd'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.847334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-04b68b6b-b08e-4625-917a-8626b7cae6bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '669187f32db34f59ac18dc23a4cbd8a3', 'request_id': 'req-04b68b6b-b08e-4625-917a-8626b7cae6bd'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.882989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-04b68b6b-b08e-4625-917a-8626b7cae6bd admin admin] The token cache is being invalidate because project 669187f32db34f59ac18dc23a4cbd8a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:42.883282 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-04b68b6b-b08e-4625-917a-8626b7cae6bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.883960 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-04b68b6b-b08e-4625-917a-8626b7cae6bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:42.885718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 933/1890] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:42 2022] DELETE /identity/v3/projects/669187f32db34f59ac18dc23a4cbd8a3 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:43.118234 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.122566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:43.150347 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3928a-5193-43c0-bb6c-709b80ebce1d admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:43.299094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:43.299942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.300763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.301413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2 admin admin] PATH_INFO: `/v3/projects/60d8cca69be34325a8e6f0abd59a1ca7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.303417 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60d8cca69be34325a8e6f0abd59a1ca7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:43.306999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:43.310820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:43.471315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60d8cca69be34325a8e6f0abd59a1ca7', 'request_id': 'req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:43.487608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60d8cca69be34325a8e6f0abd59a1ca7', 'request_id': 'req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:43.515342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60d8cca69be34325a8e6f0abd59a1ca7', 'request_id': 'req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:43.539130 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2 admin admin] The token cache is being invalidate because project 60d8cca69be34325a8e6f0abd59a1ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:43.539623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:43.541217 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-e8426a9a-3cac-472e-ada9-f7b2bf9346e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:43.546168 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 958/1891] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:43 2022] DELETE /identity/v3/projects/60d8cca69be34325a8e6f0abd59a1ca7 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:43.548396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83d9ac-a6dc-4d8e-bc31-dbce1b6caa3b None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.548691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83d9ac-a6dc-4d8e-bc31-dbce1b6caa3b None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.549434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83d9ac-a6dc-4d8e-bc31-dbce1b6caa3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.596528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-2c83d9ac-a6dc-4d8e-bc31-dbce1b6caa3b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:43.607329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2c83d9ac-a6dc-4d8e-bc31-dbce1b6caa3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:43.672426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:43.721429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 934/1892] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:43.729951 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e229cb-aa96-44c1-98d0-77f46e976840 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.730352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e229cb-aa96-44c1-98d0-77f46e976840 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.730743 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e229cb-aa96-44c1-98d0-77f46e976840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.761052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-13e229cb-aa96-44c1-98d0-77f46e976840 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:43.762825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-13e229cb-aa96-44c1-98d0-77f46e976840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:43.785447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 959/1893] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:43.786859 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83d9ac-a6dc-4d8e-bc31-dbce1b6caa3b None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.799520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83d9ac-a6dc-4d8e-bc31-dbce1b6caa3b None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:43.811500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75cf9c-3e3d-43b1-ba56-e3bfbc63cf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:43.813096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75cf9c-3e3d-43b1-ba56-e3bfbc63cf70 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.813376 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75cf9c-3e3d-43b1-ba56-e3bfbc63cf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.813647 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75cf9c-3e3d-43b1-ba56-e3bfbc63cf70 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.814358 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75cf9c-3e3d-43b1-ba56-e3bfbc63cf70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:43.816809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75cf9c-3e3d-43b1-ba56-e3bfbc63cf70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:43.818607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75cf9c-3e3d-43b1-ba56-e3bfbc63cf70 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:43.853511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 935/1894] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:43.854641 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e229cb-aa96-44c1-98d0-77f46e976840 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.864556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e229cb-aa96-44c1-98d0-77f46e976840 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:43.870198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025bcdc-41df-4e6e-89fe-f861d432e639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:43.870850 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025bcdc-41df-4e6e-89fe-f861d432e639 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.871181 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025bcdc-41df-4e6e-89fe-f861d432e639 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.871459 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025bcdc-41df-4e6e-89fe-f861d432e639 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.872134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e025bcdc-41df-4e6e-89fe-f861d432e639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:43.874036 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e025bcdc-41df-4e6e-89fe-f861d432e639 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:43.938076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 960/1895] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:43 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:43.942347 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75cf9c-3e3d-43b1-ba56-e3bfbc63cf70 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.955112 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75cf9c-3e3d-43b1-ba56-e3bfbc63cf70 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.028151 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae28a83-92d4-490d-90dc-97544d90ac26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.028789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae28a83-92d4-490d-90dc-97544d90ac26 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.029067 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae28a83-92d4-490d-90dc-97544d90ac26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.029343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae28a83-92d4-490d-90dc-97544d90ac26 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.030027 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae28a83-92d4-490d-90dc-97544d90ac26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.032035 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae28a83-92d4-490d-90dc-97544d90ac26 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.063482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 936/1896] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:43 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:44.067546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025bcdc-41df-4e6e-89fe-f861d432e639 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.083985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025bcdc-41df-4e6e-89fe-f861d432e639 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.089876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208d462-3eb8-415c-8fc6-e44892dc0fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.090925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208d462-3eb8-415c-8fc6-e44892dc0fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.091457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208d462-3eb8-415c-8fc6-e44892dc0fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.091777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208d462-3eb8-415c-8fc6-e44892dc0fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.092493 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208d462-3eb8-415c-8fc6-e44892dc0fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.094115 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208d462-3eb8-415c-8fc6-e44892dc0fc8 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.104991 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 961/1897] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:50:44.109003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae28a83-92d4-490d-90dc-97544d90ac26 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.123606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae28a83-92d4-490d-90dc-97544d90ac26 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.128544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5e3d8-b3df-44a9-9858-938f6444049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.129168 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5e3d8-b3df-44a9-9858-938f6444049f admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.129453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5e3d8-b3df-44a9-9858-938f6444049f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.129726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5e3d8-b3df-44a9-9858-938f6444049f admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.130421 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5e3d8-b3df-44a9-9858-938f6444049f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.132075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5e3d8-b3df-44a9-9858-938f6444049f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.145163 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 937/1898] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:44.150145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208d462-3eb8-415c-8fc6-e44892dc0fc8 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.161338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208d462-3eb8-415c-8fc6-e44892dc0fc8 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.167993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db7a28-9b4e-4108-bc6b-883fc7159902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.168680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db7a28-9b4e-4108-bc6b-883fc7159902 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.168969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db7a28-9b4e-4108-bc6b-883fc7159902 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.169244 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db7a28-9b4e-4108-bc6b-883fc7159902 admin admin] PATH_INFO: `/v3/projects/659c6ca446524fbe9577335b29cc2a17/users/64da49bc33864a91b6999b362f39a25f/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.189293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db7a28-9b4e-4108-bc6b-883fc7159902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=659c6ca446524fbe9577335b29cc2a17, user_id=64da49bc33864a91b6999b362f39a25f, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.192233 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db7a28-9b4e-4108-bc6b-883fc7159902 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.207571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 962/1899] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:44 2022] PUT /identity/v3/projects/659c6ca446524fbe9577335b29cc2a17/users/64da49bc33864a91b6999b362f39a25f/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:44.218267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03ce35-0f53-4bff-a547-9981a8f2a9a0 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.218663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03ce35-0f53-4bff-a547-9981a8f2a9a0 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.219012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03ce35-0f53-4bff-a547-9981a8f2a9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.252759 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-9e03ce35-0f53-4bff-a547-9981a8f2a9a0 None None] MFA Rules not processed for user `64da49bc33864a91b6999b362f39a25f`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:44.254011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9e03ce35-0f53-4bff-a547-9981a8f2a9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:44.327764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 938/1900] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:44.332444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db7a28-9b4e-4108-bc6b-883fc7159902 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.404700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db7a28-9b4e-4108-bc6b-883fc7159902 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.409778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08079e6-665b-4071-82c1-b153c09ba2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.410403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08079e6-665b-4071-82c1-b153c09ba2a7 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.410720 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08079e6-665b-4071-82c1-b153c09ba2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.410993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08079e6-665b-4071-82c1-b153c09ba2a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.411670 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08079e6-665b-4071-82c1-b153c09ba2a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.413582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08079e6-665b-4071-82c1-b153c09ba2a7 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.449999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 963/1901] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:44 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:44.451905 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03ce35-0f53-4bff-a547-9981a8f2a9a0 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.463448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03ce35-0f53-4bff-a547-9981a8f2a9a0 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.493234 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:44.538356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd82cd9-0671-4417-bf26-9946f5232037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.539321 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd82cd9-0671-4417-bf26-9946f5232037 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.539916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd82cd9-0671-4417-bf26-9946f5232037 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.540591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd82cd9-0671-4417-bf26-9946f5232037 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.541716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd82cd9-0671-4417-bf26-9946f5232037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.544275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd82cd9-0671-4417-bf26-9946f5232037 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.581702 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 939/1902] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:44 2022] POST /identity/v3/users => generated 300 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:44.586236 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08079e6-665b-4071-82c1-b153c09ba2a7 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.600257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08079e6-665b-4071-82c1-b153c09ba2a7 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.606096 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdd07f-6e37-4b61-a0ca-3ca11244a2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.606861 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdd07f-6e37-4b61-a0ca-3ca11244a2bf admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.607215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdd07f-6e37-4b61-a0ca-3ca11244a2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.607557 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdd07f-6e37-4b61-a0ca-3ca11244a2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.608338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdd07f-6e37-4b61-a0ca-3ca11244a2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.610676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdd07f-6e37-4b61-a0ca-3ca11244a2bf admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.624443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 964/1903] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:44.628317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd82cd9-0671-4417-bf26-9946f5232037 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.651455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd82cd9-0671-4417-bf26-9946f5232037 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.665161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f4543-7769-4658-b796-1b3e37754a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.666054 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f4543-7769-4658-b796-1b3e37754a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.666337 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f4543-7769-4658-b796-1b3e37754a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.666650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f4543-7769-4658-b796-1b3e37754a6c admin admin] PATH_INFO: `/v3/projects/e120aa7f71554621ac3c09b7294ed28e/users/c245431856fa44e497a8e1681f3387b8/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.690989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f4543-7769-4658-b796-1b3e37754a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e120aa7f71554621ac3c09b7294ed28e, user_id=c245431856fa44e497a8e1681f3387b8, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.693759 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f4543-7769-4658-b796-1b3e37754a6c admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.725117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 940/1904] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:44 2022] PUT /identity/v3/projects/e120aa7f71554621ac3c09b7294ed28e/users/c245431856fa44e497a8e1681f3387b8/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:44.729275 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdd07f-6e37-4b61-a0ca-3ca11244a2bf admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.736836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:44.746590 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdd07f-6e37-4b61-a0ca-3ca11244a2bf admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.841694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8bdcc-4f76-4cab-acad-800e678b7a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.842805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8bdcc-4f76-4cab-acad-800e678b7a2e admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.843682 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8bdcc-4f76-4cab-acad-800e678b7a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.844107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8bdcc-4f76-4cab-acad-800e678b7a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.845161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8bdcc-4f76-4cab-acad-800e678b7a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.847839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8bdcc-4f76-4cab-acad-800e678b7a2e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.867477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 965/1905] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:50:44.877110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f4543-7769-4658-b796-1b3e37754a6c admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:44.892853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f4543-7769-4658-b796-1b3e37754a6c admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:44.897934 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bbfc2-fd5a-4dd9-811c-6f638bd1b934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:44.898674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bbfc2-fd5a-4dd9-811c-6f638bd1b934 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.898990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bbfc2-fd5a-4dd9-811c-6f638bd1b934 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.900467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bbfc2-fd5a-4dd9-811c-6f638bd1b934 admin admin] PATH_INFO: `/v3/projects/e120aa7f71554621ac3c09b7294ed28e/users/c245431856fa44e497a8e1681f3387b8/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.904449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523bbfc2-fd5a-4dd9-811c-6f638bd1b934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e120aa7f71554621ac3c09b7294ed28e, user_id=c245431856fa44e497a8e1681f3387b8, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:44.909555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523bbfc2-fd5a-4dd9-811c-6f638bd1b934 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:44.942645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 941/1906] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:44 2022] PUT /identity/v3/projects/e120aa7f71554621ac3c09b7294ed28e/users/c245431856fa44e497a8e1681f3387b8/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:44.969588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da1823-853c-45b4-b9ba-e9bc625d8107 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.969588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da1823-853c-45b4-b9ba-e9bc625d8107 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.969588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da1823-853c-45b4-b9ba-e9bc625d8107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.004761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-38da1823-853c-45b4-b9ba-e9bc625d8107 None None] MFA Rules not processed for user `c245431856fa44e497a8e1681f3387b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:45.006067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-38da1823-853c-45b4-b9ba-e9bc625d8107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:45.114274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 966/1907] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:45.126279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bbfc2-fd5a-4dd9-811c-6f638bd1b934 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:45.128781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-523bbfc2-fd5a-4dd9-811c-6f638bd1b934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:45.202016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bbfc2-fd5a-4dd9-811c-6f638bd1b934 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:45.206202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1596f-4b5e-4a90-95d2-05f9553ce892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:45.207061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1596f-4b5e-4a90-95d2-05f9553ce892 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:45.207455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1596f-4b5e-4a90-95d2-05f9553ce892 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:45.207797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1596f-4b5e-4a90-95d2-05f9553ce892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.220103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1596f-4b5e-4a90-95d2-05f9553ce892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:45.221552 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1596f-4b5e-4a90-95d2-05f9553ce892 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:45.233611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 942/1908] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:45.972848 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da1823-853c-45b4-b9ba-e9bc625d8107 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.060070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da1823-853c-45b4-b9ba-e9bc625d8107 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.066308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.067601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.068382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.068760 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] PATH_INFO: `/v3/users/64da49bc33864a91b6999b362f39a25f` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.070606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64da49bc33864a91b6999b362f39a25f)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.075542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.147349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:46.191681 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64da49bc33864a91b6999b362f39a25f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:46.223378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64da49bc33864a91b6999b362f39a25f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:46.245756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64da49bc33864a91b6999b362f39a25f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:46.247063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64da49bc33864a91b6999b362f39a25f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:46.247849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64da49bc33864a91b6999b362f39a25f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:46.287777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 967/1909] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:45 2022] DELETE /identity/v3/users/64da49bc33864a91b6999b362f39a25f => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:46.294955 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799836d-e84a-4095-b6fb-d8b7a3c74c27 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.295702 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799836d-e84a-4095-b6fb-d8b7a3c74c27 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.295861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799836d-e84a-4095-b6fb-d8b7a3c74c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.348018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-f799836d-e84a-4095-b6fb-d8b7a3c74c27 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:46.349852 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f799836d-e84a-4095-b6fb-d8b7a3c74c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:46.492716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 943/1910] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:46.524243 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.526313 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:46.627466 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dbe6b-9466-40bf-8a48-07ca6715c396 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.631369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e010768-2e4a-48bd-9853-17a46f7420d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.632083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e010768-2e4a-48bd-9853-17a46f7420d0 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.632366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e010768-2e4a-48bd-9853-17a46f7420d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.632637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e010768-2e4a-48bd-9853-17a46f7420d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.647738 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e010768-2e4a-48bd-9853-17a46f7420d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.649328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e010768-2e4a-48bd-9853-17a46f7420d0 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.662402 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 968/1911] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:47.005513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799836d-e84a-4095-b6fb-d8b7a3c74c27 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.006958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f799836d-e84a-4095-b6fb-d8b7a3c74c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:47.021784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799836d-e84a-4095-b6fb-d8b7a3c74c27 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:47.030264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:47.030264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.030575 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.030575 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] PATH_INFO: `/v3/projects/659c6ca446524fbe9577335b29cc2a17` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.032342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] RBAC: Authorizing `identity:delete_project(project_id=659c6ca446524fbe9577335b29cc2a17)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:47.034871 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:47.053080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:47.086865 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '659c6ca446524fbe9577335b29cc2a17', 'request_id': 'req-34f33e06-3b04-4401-b63c-54bcc949510b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:47.093440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '659c6ca446524fbe9577335b29cc2a17', 'request_id': 'req-34f33e06-3b04-4401-b63c-54bcc949510b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:47.102548 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '659c6ca446524fbe9577335b29cc2a17', 'request_id': 'req-34f33e06-3b04-4401-b63c-54bcc949510b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:47.115202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] The token cache is being invalidate because project 659c6ca446524fbe9577335b29cc2a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:47.115535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:47.116146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:47.117768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 944/1912] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:47 2022] DELETE /identity/v3/projects/659c6ca446524fbe9577335b29cc2a17 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:47.123145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e010768-2e4a-48bd-9853-17a46f7420d0 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.132832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e010768-2e4a-48bd-9853-17a46f7420d0 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:47.205351 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c643641-57fd-4c89-b4f1-44592a28b460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:47.205995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c643641-57fd-4c89-b4f1-44592a28b460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.206272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c643641-57fd-4c89-b4f1-44592a28b460 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.206544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c643641-57fd-4c89-b4f1-44592a28b460 admin admin] PATH_INFO: `/v3/users/c245431856fa44e497a8e1681f3387b8` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.207770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c643641-57fd-4c89-b4f1-44592a28b460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c245431856fa44e497a8e1681f3387b8)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:47.209936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c643641-57fd-4c89-b4f1-44592a28b460 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:47.266179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c643641-57fd-4c89-b4f1-44592a28b460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c245431856fa44e497a8e1681f3387b8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:47.267430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:47.283478 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c643641-57fd-4c89-b4f1-44592a28b460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c245431856fa44e497a8e1681f3387b8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:47.292109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c643641-57fd-4c89-b4f1-44592a28b460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c245431856fa44e497a8e1681f3387b8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:47.292733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c643641-57fd-4c89-b4f1-44592a28b460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c245431856fa44e497a8e1681f3387b8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:47.293336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c643641-57fd-4c89-b4f1-44592a28b460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c245431856fa44e497a8e1681f3387b8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:47.310207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 969/1913] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:47 2022] DELETE /identity/v3/users/c245431856fa44e497a8e1681f3387b8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:47.694866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.694866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:47.731320 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f33e06-3b04-4401-b63c-54bcc949510b admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:47.850457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37d7f0-9665-4562-8219-319390fdb1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:47.851416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37d7f0-9665-4562-8219-319390fdb1c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.851993 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37d7f0-9665-4562-8219-319390fdb1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.852447 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37d7f0-9665-4562-8219-319390fdb1c7 admin admin] PATH_INFO: `/v3/projects/e120aa7f71554621ac3c09b7294ed28e` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.855083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37d7f0-9665-4562-8219-319390fdb1c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e120aa7f71554621ac3c09b7294ed28e)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:47.857264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37d7f0-9665-4562-8219-319390fdb1c7 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:47.971153 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bc37d7f0-9665-4562-8219-319390fdb1c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e120aa7f71554621ac3c09b7294ed28e', 'request_id': 'req-bc37d7f0-9665-4562-8219-319390fdb1c7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:47.983043 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bc37d7f0-9665-4562-8219-319390fdb1c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e120aa7f71554621ac3c09b7294ed28e', 'request_id': 'req-bc37d7f0-9665-4562-8219-319390fdb1c7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:47.998213 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bc37d7f0-9665-4562-8219-319390fdb1c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e120aa7f71554621ac3c09b7294ed28e', 'request_id': 'req-bc37d7f0-9665-4562-8219-319390fdb1c7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:48.019671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bc37d7f0-9665-4562-8219-319390fdb1c7 admin admin] The token cache is being invalidate because project e120aa7f71554621ac3c09b7294ed28e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:48.020696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bc37d7f0-9665-4562-8219-319390fdb1c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:48.021394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bc37d7f0-9665-4562-8219-319390fdb1c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:48.028437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 945/1914] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:47 2022] DELETE /identity/v3/projects/e120aa7f71554621ac3c09b7294ed28e => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:48.030037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2485a-6e77-4ca3-9eaf-148fe14236ef None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.030354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2485a-6e77-4ca3-9eaf-148fe14236ef None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.030664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2485a-6e77-4ca3-9eaf-148fe14236ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.076942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-81c2485a-6e77-4ca3-9eaf-148fe14236ef None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:48.078447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-81c2485a-6e77-4ca3-9eaf-148fe14236ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:48.156768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 970/1915] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:48.166675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dfae7-2580-4baa-89b0-56971c7ea2bb None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.166675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dfae7-2580-4baa-89b0-56971c7ea2bb None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.168121 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dfae7-2580-4baa-89b0-56971c7ea2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.192251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-e17dfae7-2580-4baa-89b0-56971c7ea2bb None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:48.193507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e17dfae7-2580-4baa-89b0-56971c7ea2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:48.207021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 946/1916] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:48.210835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2485a-6e77-4ca3-9eaf-148fe14236ef None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.233659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2485a-6e77-4ca3-9eaf-148fe14236ef None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.239530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04c3b4-d58d-485a-8010-0cace2547321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.242148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04c3b4-d58d-485a-8010-0cace2547321 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.242738 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04c3b4-d58d-485a-8010-0cace2547321 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.244233 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04c3b4-d58d-485a-8010-0cace2547321 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.245579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04c3b4-d58d-485a-8010-0cace2547321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:48.246078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04c3b4-d58d-485a-8010-0cace2547321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.249780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04c3b4-d58d-485a-8010-0cace2547321 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.281289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 971/1917] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:48.286108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dfae7-2580-4baa-89b0-56971c7ea2bb None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.300348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dfae7-2580-4baa-89b0-56971c7ea2bb None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.318334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806dfa6-5f35-4027-8c4d-0aac462130a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.318988 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806dfa6-5f35-4027-8c4d-0aac462130a5 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.319259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806dfa6-5f35-4027-8c4d-0aac462130a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.319529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806dfa6-5f35-4027-8c4d-0aac462130a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.320194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806dfa6-5f35-4027-8c4d-0aac462130a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.325110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806dfa6-5f35-4027-8c4d-0aac462130a5 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.398910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 947/1918] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:48 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:48.399436 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04c3b4-d58d-485a-8010-0cace2547321 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.413349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04c3b4-d58d-485a-8010-0cace2547321 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.617573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e23ff5-c551-4744-afa1-0b1bb78fdb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.623843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e23ff5-c551-4744-afa1-0b1bb78fdb0f admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.624442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e23ff5-c551-4744-afa1-0b1bb78fdb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.624768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e23ff5-c551-4744-afa1-0b1bb78fdb0f admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.625702 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e23ff5-c551-4744-afa1-0b1bb78fdb0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.627756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e23ff5-c551-4744-afa1-0b1bb78fdb0f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.669683 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 972/1919] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:48 2022] POST /identity/v3/users => generated 292 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:48.674565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806dfa6-5f35-4027-8c4d-0aac462130a5 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.688782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806dfa6-5f35-4027-8c4d-0aac462130a5 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.694417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0964be-23cf-4318-87cc-b852cdd016cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.695115 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0964be-23cf-4318-87cc-b852cdd016cf admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.695665 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0964be-23cf-4318-87cc-b852cdd016cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.695665 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0964be-23cf-4318-87cc-b852cdd016cf admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.696358 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0964be-23cf-4318-87cc-b852cdd016cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.698775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0964be-23cf-4318-87cc-b852cdd016cf admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.700492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:48.719036 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 948/1920] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:50:48.723443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e23ff5-c551-4744-afa1-0b1bb78fdb0f admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.742824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e23ff5-c551-4744-afa1-0b1bb78fdb0f admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.754756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf136995-a276-4ceb-83a8-8946b692c4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.755402 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf136995-a276-4ceb-83a8-8946b692c4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.755675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf136995-a276-4ceb-83a8-8946b692c4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.755943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf136995-a276-4ceb-83a8-8946b692c4d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.756700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf136995-a276-4ceb-83a8-8946b692c4d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.758374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf136995-a276-4ceb-83a8-8946b692c4d1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.772111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 973/1921] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:48.782342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0964be-23cf-4318-87cc-b852cdd016cf admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.797829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0964be-23cf-4318-87cc-b852cdd016cf admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.803939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3ebb3-517f-460e-81f9-e978ba40db25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.804642 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3ebb3-517f-460e-81f9-e978ba40db25 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.804920 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3ebb3-517f-460e-81f9-e978ba40db25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.805198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3ebb3-517f-460e-81f9-e978ba40db25 admin admin] PATH_INFO: `/v3/projects/1736a9bdebcf48f9aca597c4cf73e5ab/users/fbb5c5ada24f451bb988fbc46b113c09/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.822446 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3ebb3-517f-460e-81f9-e978ba40db25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1736a9bdebcf48f9aca597c4cf73e5ab, user_id=fbb5c5ada24f451bb988fbc46b113c09, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.825349 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3ebb3-517f-460e-81f9-e978ba40db25 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.843082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 949/1922] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:48 2022] PUT /identity/v3/projects/1736a9bdebcf48f9aca597c4cf73e5ab/users/fbb5c5ada24f451bb988fbc46b113c09/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:48.856185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95761c-5df6-4c5a-a8bd-4bbd1f0bc9d4 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.856623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95761c-5df6-4c5a-a8bd-4bbd1f0bc9d4 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.857385 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95761c-5df6-4c5a-a8bd-4bbd1f0bc9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.889909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-7e95761c-5df6-4c5a-a8bd-4bbd1f0bc9d4 None None] MFA Rules not processed for user `fbb5c5ada24f451bb988fbc46b113c09`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:48.891293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7e95761c-5df6-4c5a-a8bd-4bbd1f0bc9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:48.915356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:48.978920 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 974/1923] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:48.987117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3ebb3-517f-460e-81f9-e978ba40db25 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.095861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3ebb3-517f-460e-81f9-e978ba40db25 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.101004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da72fca-868d-4244-a496-92c0740c0e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.101636 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72fca-868d-4244-a496-92c0740c0e9e admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.101918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72fca-868d-4244-a496-92c0740c0e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.102190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da72fca-868d-4244-a496-92c0740c0e9e admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.103410 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da72fca-868d-4244-a496-92c0740c0e9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.106280 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da72fca-868d-4244-a496-92c0740c0e9e admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.148538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 950/1924] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:48 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:49.149330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95761c-5df6-4c5a-a8bd-4bbd1f0bc9d4 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.160956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95761c-5df6-4c5a-a8bd-4bbd1f0bc9d4 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.244586 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e6dcc-c5ae-489f-936d-bff06dd37bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.246148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e6dcc-c5ae-489f-936d-bff06dd37bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.246599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e6dcc-c5ae-489f-936d-bff06dd37bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.247165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e6dcc-c5ae-489f-936d-bff06dd37bb9 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.248666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e6dcc-c5ae-489f-936d-bff06dd37bb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.252290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e6dcc-c5ae-489f-936d-bff06dd37bb9 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.286869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 975/1925] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:49 2022] POST /identity/v3/users => generated 297 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:49.291653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da72fca-868d-4244-a496-92c0740c0e9e admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.306895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da72fca-868d-4244-a496-92c0740c0e9e admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.312239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2dff1a-9f5c-46b1-9016-b24451c0abd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.313521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dff1a-9f5c-46b1-9016-b24451c0abd4 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.313818 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dff1a-9f5c-46b1-9016-b24451c0abd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.314103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dff1a-9f5c-46b1-9016-b24451c0abd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.314866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2dff1a-9f5c-46b1-9016-b24451c0abd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.317250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2dff1a-9f5c-46b1-9016-b24451c0abd4 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.330203 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 951/1926] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:49.334362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e6dcc-c5ae-489f-936d-bff06dd37bb9 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.346895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e6dcc-c5ae-489f-936d-bff06dd37bb9 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.352163 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ac7d5-b87e-4f10-926f-9ee14a921f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.353443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ac7d5-b87e-4f10-926f-9ee14a921f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.353879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ac7d5-b87e-4f10-926f-9ee14a921f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.354160 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ac7d5-b87e-4f10-926f-9ee14a921f1d admin admin] PATH_INFO: `/v3/projects/7f956523f9cf4ec5b5b58f8f3ded2116/users/df00de88bb51461f820f799be406706e/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.393992 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ac7d5-b87e-4f10-926f-9ee14a921f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f956523f9cf4ec5b5b58f8f3ded2116, user_id=df00de88bb51461f820f799be406706e, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.397270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ac7d5-b87e-4f10-926f-9ee14a921f1d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.418700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 976/1927] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:49 2022] PUT /identity/v3/projects/7f956523f9cf4ec5b5b58f8f3ded2116/users/df00de88bb51461f820f799be406706e/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:49.422596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2dff1a-9f5c-46b1-9016-b24451c0abd4 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.435110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2dff1a-9f5c-46b1-9016-b24451c0abd4 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.531753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9869f7-c068-4620-9f64-9665f791b3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.532402 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9869f7-c068-4620-9f64-9665f791b3db admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.532694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9869f7-c068-4620-9f64-9665f791b3db admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.533071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9869f7-c068-4620-9f64-9665f791b3db admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.533788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9869f7-c068-4620-9f64-9665f791b3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.535515 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9869f7-c068-4620-9f64-9665f791b3db admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.553719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 952/1928] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:49.554488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ac7d5-b87e-4f10-926f-9ee14a921f1d admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.576767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ac7d5-b87e-4f10-926f-9ee14a921f1d admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.586071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543c99d-d31c-40d7-8a5e-ce584d6364f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.586902 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543c99d-d31c-40d7-8a5e-ce584d6364f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.587873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543c99d-d31c-40d7-8a5e-ce584d6364f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.588420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543c99d-d31c-40d7-8a5e-ce584d6364f4 admin admin] PATH_INFO: `/v3/projects/7f956523f9cf4ec5b5b58f8f3ded2116/users/df00de88bb51461f820f799be406706e/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.593054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543c99d-d31c-40d7-8a5e-ce584d6364f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f956523f9cf4ec5b5b58f8f3ded2116, user_id=df00de88bb51461f820f799be406706e, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.596673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543c99d-d31c-40d7-8a5e-ce584d6364f4 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.623635 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 977/1929] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:49 2022] PUT /identity/v3/projects/7f956523f9cf4ec5b5b58f8f3ded2116/users/df00de88bb51461f820f799be406706e/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:49.628882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110e1a8-d4e3-47e0-a42d-30283fbbf78b None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.629248 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110e1a8-d4e3-47e0-a42d-30283fbbf78b None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.629599 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110e1a8-d4e3-47e0-a42d-30283fbbf78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.685558 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-a110e1a8-d4e3-47e0-a42d-30283fbbf78b None None] MFA Rules not processed for user `df00de88bb51461f820f799be406706e`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:49.687616 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a110e1a8-d4e3-47e0-a42d-30283fbbf78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:49.731165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:49.824327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 953/1930] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:49.841753 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543c99d-d31c-40d7-8a5e-ce584d6364f4 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.843666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1543c99d-d31c-40d7-8a5e-ce584d6364f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:49.936612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543c99d-d31c-40d7-8a5e-ce584d6364f4 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.953346 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a89c7-1c44-401f-b602-e37305ec7602 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.954446 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a89c7-1c44-401f-b602-e37305ec7602 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.954970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a89c7-1c44-401f-b602-e37305ec7602 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.955457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a89c7-1c44-401f-b602-e37305ec7602 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.968256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a89c7-1c44-401f-b602-e37305ec7602 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.969752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a89c7-1c44-401f-b602-e37305ec7602 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.984168 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 978/1931] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:50.115873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110e1a8-d4e3-47e0-a42d-30283fbbf78b None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:50.127562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110e1a8-d4e3-47e0-a42d-30283fbbf78b None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:50.131906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c858dcc-8ae6-469d-a197-0bd2a09ab8e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:50.132510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c858dcc-8ae6-469d-a197-0bd2a09ab8e5 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:50.132820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c858dcc-8ae6-469d-a197-0bd2a09ab8e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:50.133093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c858dcc-8ae6-469d-a197-0bd2a09ab8e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:50.221453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c858dcc-8ae6-469d-a197-0bd2a09ab8e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:50.223245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c858dcc-8ae6-469d-a197-0bd2a09ab8e5 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:50.239743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 954/1932] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:50.868307 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a89c7-1c44-401f-b602-e37305ec7602 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:50.891368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:50.987722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a89c7-1c44-401f-b602-e37305ec7602 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:50.993615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6542b2-fb9d-4b1e-b112-eeb6013bdc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:50.994447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6542b2-fb9d-4b1e-b112-eeb6013bdc80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:50.994808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6542b2-fb9d-4b1e-b112-eeb6013bdc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:50.995800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6542b2-fb9d-4b1e-b112-eeb6013bdc80 admin admin] PATH_INFO: `/v3/users/fbb5c5ada24f451bb988fbc46b113c09` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:50.997175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6542b2-fb9d-4b1e-b112-eeb6013bdc80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbb5c5ada24f451bb988fbc46b113c09)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:51.000599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6542b2-fb9d-4b1e-b112-eeb6013bdc80 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:51.097846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9a6542b2-fb9d-4b1e-b112-eeb6013bdc80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbb5c5ada24f451bb988fbc46b113c09'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:51.120307 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9a6542b2-fb9d-4b1e-b112-eeb6013bdc80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbb5c5ada24f451bb988fbc46b113c09'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:51.132297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9a6542b2-fb9d-4b1e-b112-eeb6013bdc80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbb5c5ada24f451bb988fbc46b113c09'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:51.134014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9a6542b2-fb9d-4b1e-b112-eeb6013bdc80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbb5c5ada24f451bb988fbc46b113c09'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:51.136075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9a6542b2-fb9d-4b1e-b112-eeb6013bdc80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbb5c5ada24f451bb988fbc46b113c09'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:51.139954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d3f37-dc77-4ec7-8b0f-a8ef67fa4757 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.140471 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d3f37-dc77-4ec7-8b0f-a8ef67fa4757 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.140581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d3f37-dc77-4ec7-8b0f-a8ef67fa4757 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.142199 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 955/1933] 10.222.0.86 () {54 vars in 921 bytes} [Fri Aug 5 19:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:51.156647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 979/1934] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:50 2022] DELETE /identity/v3/users/fbb5c5ada24f451bb988fbc46b113c09 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:51.158642 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003f3a-dbd2-4770-bc20-cf8b73e4bec9 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.158766 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003f3a-dbd2-4770-bc20-cf8b73e4bec9 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.159072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea003f3a-dbd2-4770-bc20-cf8b73e4bec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.164740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e8b78-2a64-4c96-88aa-00e3ade35cbe None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.165146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e8b78-2a64-4c96-88aa-00e3ade35cbe None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.167345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e8b78-2a64-4c96-88aa-00e3ade35cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.197145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-ea003f3a-dbd2-4770-bc20-cf8b73e4bec9 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:51.201636 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ea003f3a-dbd2-4770-bc20-cf8b73e4bec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:51.208189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c12e8b78-2a64-4c96-88aa-00e3ade35cbe None None] MFA Rules not processed for user `cce71f93c80d433b830c448d0a7a14e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:51.209425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c12e8b78-2a64-4c96-88aa-00e3ade35cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:51.360932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 956/1935] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:51.371420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003f3a-dbd2-4770-bc20-cf8b73e4bec9 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:51.373278 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ea003f3a-dbd2-4770-bc20-cf8b73e4bec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:51.414412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 980/1936] 10.222.0.86 () {58 vars in 1018 bytes} [Fri Aug 5 19:50:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:51.437230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e8b78-2a64-4c96-88aa-00e3ade35cbe None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:51.439566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c12e8b78-2a64-4c96-88aa-00e3ade35cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:51.466991 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e8b78-2a64-4c96-88aa-00e3ade35cbe None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:51.496642 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d402c3d-e5b9-41e4-96e6-5dee448491dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:51.497356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d402c3d-e5b9-41e4-96e6-5dee448491dc service placement] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.499856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d402c3d-e5b9-41e4-96e6-5dee448491dc service placement] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.500143 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d402c3d-e5b9-41e4-96e6-5dee448491dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.516404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea003f3a-dbd2-4770-bc20-cf8b73e4bec9 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:51.523981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d402c3d-e5b9-41e4-96e6-5dee448491dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:51.528166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd583223-05b4-41de-acc5-43f52b1d4bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:51.528526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d402c3d-e5b9-41e4-96e6-5dee448491dc service placement] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:51.529369 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd583223-05b4-41de-acc5-43f52b1d4bcf service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.529874 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd583223-05b4-41de-acc5-43f52b1d4bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.530475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd583223-05b4-41de-acc5-43f52b1d4bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.542216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd583223-05b4-41de-acc5-43f52b1d4bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:51.543831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd583223-05b4-41de-acc5-43f52b1d4bcf service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:51.555076 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 957/1937] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:51.561712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 981/1938] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:52.026421 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd583223-05b4-41de-acc5-43f52b1d4bcf service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:52.028762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-dd583223-05b4-41de-acc5-43f52b1d4bcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:52.063770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd583223-05b4-41de-acc5-43f52b1d4bcf service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:52.070365 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c8486-98af-48aa-b581-9d3fda149407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:52.070950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c8486-98af-48aa-b581-9d3fda149407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:52.071937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c8486-98af-48aa-b581-9d3fda149407 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:52.073152 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c8486-98af-48aa-b581-9d3fda149407 admin admin] PATH_INFO: `/v3/projects/1736a9bdebcf48f9aca597c4cf73e5ab` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:52.075079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c8486-98af-48aa-b581-9d3fda149407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1736a9bdebcf48f9aca597c4cf73e5ab)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:52.077037 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c8486-98af-48aa-b581-9d3fda149407 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:52.079784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:52.150603 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-0c0c8486-98af-48aa-b581-9d3fda149407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1736a9bdebcf48f9aca597c4cf73e5ab', 'request_id': 'req-0c0c8486-98af-48aa-b581-9d3fda149407'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.171690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-0c0c8486-98af-48aa-b581-9d3fda149407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1736a9bdebcf48f9aca597c4cf73e5ab', 'request_id': 'req-0c0c8486-98af-48aa-b581-9d3fda149407'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.185960 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-0c0c8486-98af-48aa-b581-9d3fda149407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1736a9bdebcf48f9aca597c4cf73e5ab', 'request_id': 'req-0c0c8486-98af-48aa-b581-9d3fda149407'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.204013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-0c0c8486-98af-48aa-b581-9d3fda149407 admin admin] The token cache is being invalidate because project 1736a9bdebcf48f9aca597c4cf73e5ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:52.204311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-0c0c8486-98af-48aa-b581-9d3fda149407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.205000 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-0c0c8486-98af-48aa-b581-9d3fda149407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.208429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 958/1939] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:52 2022] DELETE /identity/v3/projects/1736a9bdebcf48f9aca597c4cf73e5ab => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:52.213747 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d402c3d-e5b9-41e4-96e6-5dee448491dc service placement] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:52.225645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d402c3d-e5b9-41e4-96e6-5dee448491dc service placement] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:52.241704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:52.302300 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07130d-8bc5-4754-b196-1eb8d0885201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:52.303092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07130d-8bc5-4754-b196-1eb8d0885201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:52.303585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07130d-8bc5-4754-b196-1eb8d0885201 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:52.303924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07130d-8bc5-4754-b196-1eb8d0885201 admin admin] PATH_INFO: `/v3/users/df00de88bb51461f820f799be406706e` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:52.305478 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07130d-8bc5-4754-b196-1eb8d0885201 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df00de88bb51461f820f799be406706e)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:52.308373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07130d-8bc5-4754-b196-1eb8d0885201 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:52.406122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3d07130d-8bc5-4754-b196-1eb8d0885201 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df00de88bb51461f820f799be406706e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.424606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3d07130d-8bc5-4754-b196-1eb8d0885201 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df00de88bb51461f820f799be406706e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.436721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3d07130d-8bc5-4754-b196-1eb8d0885201 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df00de88bb51461f820f799be406706e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.437853 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3d07130d-8bc5-4754-b196-1eb8d0885201 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df00de88bb51461f820f799be406706e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.438722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3d07130d-8bc5-4754-b196-1eb8d0885201 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df00de88bb51461f820f799be406706e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.459175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 982/1940] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:52 2022] DELETE /identity/v3/users/df00de88bb51461f820f799be406706e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:52.516689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3eca5-9d05-4951-b18a-e8f82600e293 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:52.516905 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3eca5-9d05-4951-b18a-e8f82600e293 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:52.517164 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3eca5-9d05-4951-b18a-e8f82600e293 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:52.518551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 959/1941] 10.222.0.86 () {54 vars in 921 bytes} [Fri Aug 5 19:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:52.566562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a74823-2c3c-4ff2-acd5-e600543a0c3f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:52.566815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a74823-2c3c-4ff2-acd5-e600543a0c3f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:52.567092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a74823-2c3c-4ff2-acd5-e600543a0c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:52.601324 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-48a74823-2c3c-4ff2-acd5-e600543a0c3f None None] MFA Rules not processed for user `cce71f93c80d433b830c448d0a7a14e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:52.603638 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-48a74823-2c3c-4ff2-acd5-e600543a0c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:52.663062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3eca5-9d05-4951-b18a-e8f82600e293 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:52.666602 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-fba3eca5-9d05-4951-b18a-e8f82600e293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:52.759651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 983/1942] 10.222.0.86 () {58 vars in 1018 bytes} [Fri Aug 5 19:50:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:52.785831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a74823-2c3c-4ff2-acd5-e600543a0c3f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:52.787298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-48a74823-2c3c-4ff2-acd5-e600543a0c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:52.791472 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3eca5-9d05-4951-b18a-e8f82600e293 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:52.797471 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f576ec8-7be6-444e-8260-cf07fd565a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:52.798244 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f576ec8-7be6-444e-8260-cf07fd565a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:52.798528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f576ec8-7be6-444e-8260-cf07fd565a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:52.798840 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f576ec8-7be6-444e-8260-cf07fd565a43 admin admin] PATH_INFO: `/v3/users/4373be2e40c940a89a3c3c6f4f306b14` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:52.800118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f576ec8-7be6-444e-8260-cf07fd565a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4373be2e40c940a89a3c3c6f4f306b14)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:52.802593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f576ec8-7be6-444e-8260-cf07fd565a43 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:52.802971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a74823-2c3c-4ff2-acd5-e600543a0c3f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:52.811316 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a711545-bbe6-44d0-93cc-69fb2392be39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:52.812867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a711545-bbe6-44d0-93cc-69fb2392be39 service placement] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:52.812867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a711545-bbe6-44d0-93cc-69fb2392be39 service placement] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:52.812867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a711545-bbe6-44d0-93cc-69fb2392be39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:52.823850 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a711545-bbe6-44d0-93cc-69fb2392be39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:52.825681 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a711545-bbe6-44d0-93cc-69fb2392be39 service placement] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:52.839985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 984/1943] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:50:52.899535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8f576ec8-7be6-444e-8260-cf07fd565a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4373be2e40c940a89a3c3c6f4f306b14'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.910551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8f576ec8-7be6-444e-8260-cf07fd565a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4373be2e40c940a89a3c3c6f4f306b14'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.911984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8f576ec8-7be6-444e-8260-cf07fd565a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4373be2e40c940a89a3c3c6f4f306b14'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.929258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8f576ec8-7be6-444e-8260-cf07fd565a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4373be2e40c940a89a3c3c6f4f306b14'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.940714 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8f576ec8-7be6-444e-8260-cf07fd565a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4373be2e40c940a89a3c3c6f4f306b14'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:52.966831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 960/1944] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:52 2022] DELETE /identity/v3/users/4373be2e40c940a89a3c3c6f4f306b14 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:53.021526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a711545-bbe6-44d0-93cc-69fb2392be39 service placement] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:53.023294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8a711545-bbe6-44d0-93cc-69fb2392be39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:53.046749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a711545-bbe6-44d0-93cc-69fb2392be39 service placement] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:53.074327 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:53.163416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5310f7-f3bf-4cee-a124-0496e453106b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:53.164115 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5310f7-f3bf-4cee-a124-0496e453106b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.166935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5310f7-f3bf-4cee-a124-0496e453106b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.167522 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5310f7-f3bf-4cee-a124-0496e453106b admin admin] PATH_INFO: `/v3/projects/7f956523f9cf4ec5b5b58f8f3ded2116` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.170018 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5310f7-f3bf-4cee-a124-0496e453106b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f956523f9cf4ec5b5b58f8f3ded2116)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:53.173056 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5310f7-f3bf-4cee-a124-0496e453106b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:53.251861 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bb5310f7-f3bf-4cee-a124-0496e453106b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f956523f9cf4ec5b5b58f8f3ded2116', 'request_id': 'req-bb5310f7-f3bf-4cee-a124-0496e453106b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:53.259791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bb5310f7-f3bf-4cee-a124-0496e453106b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f956523f9cf4ec5b5b58f8f3ded2116', 'request_id': 'req-bb5310f7-f3bf-4cee-a124-0496e453106b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:53.271651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bb5310f7-f3bf-4cee-a124-0496e453106b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f956523f9cf4ec5b5b58f8f3ded2116', 'request_id': 'req-bb5310f7-f3bf-4cee-a124-0496e453106b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:53.289768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bb5310f7-f3bf-4cee-a124-0496e453106b admin admin] The token cache is being invalidate because project 7f956523f9cf4ec5b5b58f8f3ded2116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:53.290400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bb5310f7-f3bf-4cee-a124-0496e453106b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:53.291834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bb5310f7-f3bf-4cee-a124-0496e453106b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:53.297756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 985/1945] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:53 2022] DELETE /identity/v3/projects/7f956523f9cf4ec5b5b58f8f3ded2116 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:53.303940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284cb6-dc50-4896-acc8-da75e87f8cdc None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.304480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284cb6-dc50-4896-acc8-da75e87f8cdc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.304480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9284cb6-dc50-4896-acc8-da75e87f8cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.329680 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c9284cb6-dc50-4896-acc8-da75e87f8cdc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:53.331203 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c9284cb6-dc50-4896-acc8-da75e87f8cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:53.337644 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:53.438881 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e4c47-26c2-4158-93a1-25bdbc30ae7a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.439276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e4c47-26c2-4158-93a1-25bdbc30ae7a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.439613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e4c47-26c2-4158-93a1-25bdbc30ae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.443891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 961/1946] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:53.463342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-5d5e4c47-26c2-4158-93a1-25bdbc30ae7a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:53.464696 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5d5e4c47-26c2-4158-93a1-25bdbc30ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:53.487497 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 986/1947] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:53.488618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9284cb6-dc50-4896-acc8-da75e87f8cdc None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:53.519082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9284cb6-dc50-4896-acc8-da75e87f8cdc None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:53.534547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e4c47-26c2-4158-93a1-25bdbc30ae7a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:53.537253 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5d5e4c47-26c2-4158-93a1-25bdbc30ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:53.541347 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8e515-0d6d-4afa-b174-52af85a9a164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:53.542225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8e515-0d6d-4afa-b174-52af85a9a164 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.542558 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8e515-0d6d-4afa-b174-52af85a9a164 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.543000 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8e515-0d6d-4afa-b174-52af85a9a164 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.544532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8e515-0d6d-4afa-b174-52af85a9a164 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:53.544532 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8e515-0d6d-4afa-b174-52af85a9a164 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:53.547898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8e515-0d6d-4afa-b174-52af85a9a164 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:53.569131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e4c47-26c2-4158-93a1-25bdbc30ae7a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:53.581656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 962/1948] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:53.582681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8e515-0d6d-4afa-b174-52af85a9a164 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:53.583856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:53.586223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.586930 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.588060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] PATH_INFO: `/v3/projects/6d86ae2978c34fcaacdaf63eca3706f8` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.590795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d86ae2978c34fcaacdaf63eca3706f8)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:53.593490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:53.595657 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8e515-0d6d-4afa-b174-52af85a9a164 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:53.607345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1109c-3cdd-4013-a533-ec990613b514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:53.607974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1109c-3cdd-4013-a533-ec990613b514 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.608252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1109c-3cdd-4013-a533-ec990613b514 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.608524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1109c-3cdd-4013-a533-ec990613b514 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.609200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1109c-3cdd-4013-a533-ec990613b514 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:53.611333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1109c-3cdd-4013-a533-ec990613b514 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:53.655685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 963/1949] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:53 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:53.659857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1109c-3cdd-4013-a533-ec990613b514 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:53.691085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d86ae2978c34fcaacdaf63eca3706f8', 'request_id': 'req-51bb3b50-2b0b-4c06-82ad-33355f41cea0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:53.703809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d86ae2978c34fcaacdaf63eca3706f8', 'request_id': 'req-51bb3b50-2b0b-4c06-82ad-33355f41cea0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:53.720183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d86ae2978c34fcaacdaf63eca3706f8', 'request_id': 'req-51bb3b50-2b0b-4c06-82ad-33355f41cea0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:53.735134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] The token cache is being invalidate because project 6d86ae2978c34fcaacdaf63eca3706f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:53.735426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:53.736359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:53.738252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 987/1950] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:53 2022] DELETE /identity/v3/projects/6d86ae2978c34fcaacdaf63eca3706f8 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:53.741973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:53.746027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:53.770679 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1109c-3cdd-4013-a533-ec990613b514 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:53.790776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:53.847493 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb3b50-2b0b-4c06-82ad-33355f41cea0 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:53.853770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:53.854763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.855328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.855761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] PATH_INFO: `/v3/users/8902773db06844279438b875ab727425` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.858041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8902773db06844279438b875ab727425)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:53.861325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:53.964148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8902773db06844279438b875ab727425'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:53.974885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef3376-598a-4e27-9682-d0792b9b7fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:53.975844 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef3376-598a-4e27-9682-d0792b9b7fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:53.975844 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef3376-598a-4e27-9682-d0792b9b7fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:53.976028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef3376-598a-4e27-9682-d0792b9b7fb7 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:53.976785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef3376-598a-4e27-9682-d0792b9b7fb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:53.979868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef3376-598a-4e27-9682-d0792b9b7fb7 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.018463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8902773db06844279438b875ab727425'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:54.029142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 964/1951] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:53 2022] POST /identity/v3/users => generated 288 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:54.033341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8902773db06844279438b875ab727425'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:54.033478 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef3376-598a-4e27-9682-d0792b9b7fb7 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.034874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8902773db06844279438b875ab727425'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:54.035502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-79ef3376-598a-4e27-9682-d0792b9b7fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:54.035725 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8902773db06844279438b875ab727425'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:54.052966 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef3376-598a-4e27-9682-d0792b9b7fb7 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.059426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92ebe8-6483-414d-99c7-141e6453d451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.060058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92ebe8-6483-414d-99c7-141e6453d451 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.060333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92ebe8-6483-414d-99c7-141e6453d451 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.060785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92ebe8-6483-414d-99c7-141e6453d451 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.061648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92ebe8-6483-414d-99c7-141e6453d451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.066296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92ebe8-6483-414d-99c7-141e6453d451 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.070302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 988/1952] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:50:53 2022] DELETE /identity/v3/users/8902773db06844279438b875ab727425 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:54.089798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 965/1953] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:54.094506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.097891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:54.251011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5a49b-ad61-4e88-bd4b-b94f6c9cadfe admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.264472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415300f-1e98-4ac0-977d-807f70d46283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.266602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415300f-1e98-4ac0-977d-807f70d46283 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.266922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415300f-1e98-4ac0-977d-807f70d46283 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.268357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415300f-1e98-4ac0-977d-807f70d46283 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.269248 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415300f-1e98-4ac0-977d-807f70d46283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.271670 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415300f-1e98-4ac0-977d-807f70d46283 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.291791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 989/1954] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:54.293361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92ebe8-6483-414d-99c7-141e6453d451 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.299299 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:54.311084 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92ebe8-6483-414d-99c7-141e6453d451 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.320552 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e13547-e2ed-4956-b5a7-83b209d5a594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.322217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e13547-e2ed-4956-b5a7-83b209d5a594 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.322923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e13547-e2ed-4956-b5a7-83b209d5a594 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.323426 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e13547-e2ed-4956-b5a7-83b209d5a594 admin admin] PATH_INFO: `/v3/projects/6bced703b1fa483492888f907d4b5c68/users/fb082a67f98d4b12bcf5c8f97fff2cfd/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.375342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e13547-e2ed-4956-b5a7-83b209d5a594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bced703b1fa483492888f907d4b5c68, user_id=fb082a67f98d4b12bcf5c8f97fff2cfd, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.375820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415300f-1e98-4ac0-977d-807f70d46283 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.378379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4415300f-1e98-4ac0-977d-807f70d46283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:54.384137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e13547-e2ed-4956-b5a7-83b209d5a594 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.409509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 966/1955] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:54 2022] PUT /identity/v3/projects/6bced703b1fa483492888f907d4b5c68/users/fb082a67f98d4b12bcf5c8f97fff2cfd/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:54.420552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415300f-1e98-4ac0-977d-807f70d46283 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.423519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef6676-82f6-441b-927a-e2824df7834a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.424129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef6676-82f6-441b-927a-e2824df7834a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.424800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef6676-82f6-441b-927a-e2824df7834a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.471752 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-71ef6676-82f6-441b-927a-e2824df7834a None None] MFA Rules not processed for user `fb082a67f98d4b12bcf5c8f97fff2cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:54.474696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-71ef6676-82f6-441b-927a-e2824df7834a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:54.545488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:54.594510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 967/1956] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:54.600783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.600783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.600783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.600783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21 admin admin] PATH_INFO: `/v3/projects/2f5fcc7816fa47fc90c10cce190e3017` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.612479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f5fcc7816fa47fc90c10cce190e3017)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.614735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef6676-82f6-441b-927a-e2824df7834a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.615517 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.617748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-71ef6676-82f6-441b-927a-e2824df7834a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:54.697264 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f5fcc7816fa47fc90c10cce190e3017', 'request_id': 'req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:54.703308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef6676-82f6-441b-927a-e2824df7834a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.706140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f5fcc7816fa47fc90c10cce190e3017', 'request_id': 'req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:54.724448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f5fcc7816fa47fc90c10cce190e3017', 'request_id': 'req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:54.743979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21 admin admin] The token cache is being invalidate because project 2f5fcc7816fa47fc90c10cce190e3017 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:50:54.744178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:54.745054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f8d70e9e-e5ce-4813-bb8b-cdb457a33f21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:50:54.747466 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 990/1957] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:50:54 2022] DELETE /identity/v3/projects/2f5fcc7816fa47fc90c10cce190e3017 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:54.752190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908d81a-8083-4427-9f30-58fddbe34f13 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.752414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908d81a-8083-4427-9f30-58fddbe34f13 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.752664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908d81a-8083-4427-9f30-58fddbe34f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.784295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d9f88-aa25-42af-aca0-638e4914b18c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.784904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d9f88-aa25-42af-aca0-638e4914b18c service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.785188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d9f88-aa25-42af-aca0-638e4914b18c service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.785460 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d9f88-aa25-42af-aca0-638e4914b18c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.787345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5d2d9f88-aa25-42af-aca0-638e4914b18c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:54.799992 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-4908d81a-8083-4427-9f30-58fddbe34f13 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:54.801340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4908d81a-8083-4427-9f30-58fddbe34f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:54.872652 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:54.893136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 991/1958] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:54.901691 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa45c87-1521-478f-bdbf-fce8890982a6 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.901977 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa45c87-1521-478f-bdbf-fce8890982a6 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.902259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa45c87-1521-478f-bdbf-fce8890982a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.908019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d9f88-aa25-42af-aca0-638e4914b18c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.909491 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d9f88-aa25-42af-aca0-638e4914b18c service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.923902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 968/1959] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:50:54.926770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-4aa45c87-1521-478f-bdbf-fce8890982a6 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:54.928027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4aa45c87-1521-478f-bdbf-fce8890982a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:54.937368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 992/1960] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:50:54.941992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d9f88-aa25-42af-aca0-638e4914b18c service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.967924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d9f88-aa25-42af-aca0-638e4914b18c service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.979853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba0b0f-5e4d-44d3-887a-71d8347e7ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.980363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba0b0f-5e4d-44d3-887a-71d8347e7ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.980645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba0b0f-5e4d-44d3-887a-71d8347e7ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.980912 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba0b0f-5e4d-44d3-887a-71d8347e7ff4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.981607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba0b0f-5e4d-44d3-887a-71d8347e7ff4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:54.981909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba0b0f-5e4d-44d3-887a-71d8347e7ff4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.986325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba0b0f-5e4d-44d3-887a-71d8347e7ff4 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.020634 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 969/1961] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:55.026857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa45c87-1521-478f-bdbf-fce8890982a6 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:55.038147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa45c87-1521-478f-bdbf-fce8890982a6 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.044098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbed3a6-e4fd-40da-b0ba-da63b9848be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.044993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbed3a6-e4fd-40da-b0ba-da63b9848be3 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.045354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbed3a6-e4fd-40da-b0ba-da63b9848be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.045696 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbed3a6-e4fd-40da-b0ba-da63b9848be3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.046446 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbed3a6-e4fd-40da-b0ba-da63b9848be3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.048695 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbed3a6-e4fd-40da-b0ba-da63b9848be3 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.137751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 993/1962] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:50:55 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:55.142782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba0b0f-5e4d-44d3-887a-71d8347e7ff4 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:55.166692 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba0b0f-5e4d-44d3-887a-71d8347e7ff4 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.360108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa12d11-4fea-482f-a947-243e08a24172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.360803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa12d11-4fea-482f-a947-243e08a24172 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.361010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa12d11-4fea-482f-a947-243e08a24172 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.361286 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa12d11-4fea-482f-a947-243e08a24172 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.362597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa12d11-4fea-482f-a947-243e08a24172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.364591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa12d11-4fea-482f-a947-243e08a24172 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.409514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 970/1963] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:55 2022] POST /identity/v3/users => generated 289 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:55.413682 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbed3a6-e4fd-40da-b0ba-da63b9848be3 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:55.435925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbed3a6-e4fd-40da-b0ba-da63b9848be3 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.441995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb6f52-9d5f-4a5a-869a-8434783c52f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.442668 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb6f52-9d5f-4a5a-869a-8434783c52f2 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.442979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb6f52-9d5f-4a5a-869a-8434783c52f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.443443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb6f52-9d5f-4a5a-869a-8434783c52f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.444289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb6f52-9d5f-4a5a-869a-8434783c52f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.445986 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb6f52-9d5f-4a5a-869a-8434783c52f2 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.466564 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa12d11-4fea-482f-a947-243e08a24172 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:55.466935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 994/1964] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:55.483631 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa12d11-4fea-482f-a947-243e08a24172 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.496792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097a525-f34b-4b13-9ea0-991e3a483532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.497433 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097a525-f34b-4b13-9ea0-991e3a483532 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.498163 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097a525-f34b-4b13-9ea0-991e3a483532 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.498163 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097a525-f34b-4b13-9ea0-991e3a483532 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.498811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097a525-f34b-4b13-9ea0-991e3a483532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.500693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097a525-f34b-4b13-9ea0-991e3a483532 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.530050 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 971/1965] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:50:55.531434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb6f52-9d5f-4a5a-869a-8434783c52f2 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:55.551469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb6f52-9d5f-4a5a-869a-8434783c52f2 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.565629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eeb57a-780a-4172-af61-34ad2a5070a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.566462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eeb57a-780a-4172-af61-34ad2a5070a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.567370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eeb57a-780a-4172-af61-34ad2a5070a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.567663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eeb57a-780a-4172-af61-34ad2a5070a2 admin admin] PATH_INFO: `/v3/projects/02c6b82d3ad74e0caa781f6184b67c42/users/79a114faf69241c585dcaf3ec3c1c066/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.605013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eeb57a-780a-4172-af61-34ad2a5070a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c6b82d3ad74e0caa781f6184b67c42, user_id=79a114faf69241c585dcaf3ec3c1c066, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.608025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eeb57a-780a-4172-af61-34ad2a5070a2 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.641581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 995/1966] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:50:55 2022] PUT /identity/v3/projects/02c6b82d3ad74e0caa781f6184b67c42/users/79a114faf69241c585dcaf3ec3c1c066/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:55.645775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b78174-5531-4a94-989c-329ad9ed2ca9 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.646045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b78174-5531-4a94-989c-329ad9ed2ca9 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.646315 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b78174-5531-4a94-989c-329ad9ed2ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.711404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c6b78174-5531-4a94-989c-329ad9ed2ca9 None None] MFA Rules not processed for user `79a114faf69241c585dcaf3ec3c1c066`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:55.713344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c6b78174-5531-4a94-989c-329ad9ed2ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:55.801663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 972/1967] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:50:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:50:55.815607 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eeb57a-780a-4172-af61-34ad2a5070a2 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:55.817488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-88eeb57a-780a-4172-af61-34ad2a5070a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:55.833604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:55.919593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eeb57a-780a-4172-af61-34ad2a5070a2 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.926674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489aa3c2-1dec-4569-b5ec-7528ee60adaf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72d4459164844e718544d7fde333db76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.926674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489aa3c2-1dec-4569-b5ec-7528ee60adaf service glance] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.926674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489aa3c2-1dec-4569-b5ec-7528ee60adaf service glance] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.926674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489aa3c2-1dec-4569-b5ec-7528ee60adaf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.937885 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489aa3c2-1dec-4569-b5ec-7528ee60adaf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.939674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489aa3c2-1dec-4569-b5ec-7528ee60adaf service glance] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.955047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 996/1968] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:50:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:51:05.797744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:05.947833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=31 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:12.340837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b78174-5531-4a94-989c-329ad9ed2ca9 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:12.342308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c6b78174-5531-4a94-989c-329ad9ed2ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:12.415859 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b78174-5531-4a94-989c-329ad9ed2ca9 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:12.420407 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5811f5-285f-4b60-bcaa-45fdf04c01cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:12.421049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5811f5-285f-4b60-bcaa-45fdf04c01cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.421330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5811f5-285f-4b60-bcaa-45fdf04c01cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.421603 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5811f5-285f-4b60-bcaa-45fdf04c01cb admin admin] PATH_INFO: `/v3/users/a1d7a67f595b44a89abe80bc4f714821` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.422815 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5811f5-285f-4b60-bcaa-45fdf04c01cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d7a67f595b44a89abe80bc4f714821)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:12.424836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5811f5-285f-4b60-bcaa-45fdf04c01cb admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:12.486514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ae5811f5-285f-4b60-bcaa-45fdf04c01cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d7a67f595b44a89abe80bc4f714821'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:12.496791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ae5811f5-285f-4b60-bcaa-45fdf04c01cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d7a67f595b44a89abe80bc4f714821'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:12.497523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ae5811f5-285f-4b60-bcaa-45fdf04c01cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d7a67f595b44a89abe80bc4f714821'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:12.510774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ae5811f5-285f-4b60-bcaa-45fdf04c01cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d7a67f595b44a89abe80bc4f714821'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:12.522685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ae5811f5-285f-4b60-bcaa-45fdf04c01cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d7a67f595b44a89abe80bc4f714821'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:12.532095 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 973/1969] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:51:12 2022] DELETE /identity/v3/users/a1d7a67f595b44a89abe80bc4f714821 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:51:12.741256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489aa3c2-1dec-4569-b5ec-7528ee60adaf service glance] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:12.756078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489aa3c2-1dec-4569-b5ec-7528ee60adaf service glance] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:12.817801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411b2bc-8665-4147-805a-5ae301abf225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:12.818451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411b2bc-8665-4147-805a-5ae301abf225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.818759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411b2bc-8665-4147-805a-5ae301abf225 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.819025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411b2bc-8665-4147-805a-5ae301abf225 admin admin] PATH_INFO: `/v3/projects/d773fbf1acca4df9947f053573486931` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.820250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411b2bc-8665-4147-805a-5ae301abf225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d773fbf1acca4df9947f053573486931)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:12.823206 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411b2bc-8665-4147-805a-5ae301abf225 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:12.881151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3411b2bc-8665-4147-805a-5ae301abf225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd773fbf1acca4df9947f053573486931', 'request_id': 'req-3411b2bc-8665-4147-805a-5ae301abf225'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:12.889647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3411b2bc-8665-4147-805a-5ae301abf225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd773fbf1acca4df9947f053573486931', 'request_id': 'req-3411b2bc-8665-4147-805a-5ae301abf225'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:12.898973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3411b2bc-8665-4147-805a-5ae301abf225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd773fbf1acca4df9947f053573486931', 'request_id': 'req-3411b2bc-8665-4147-805a-5ae301abf225'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:12.912548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3411b2bc-8665-4147-805a-5ae301abf225 admin admin] The token cache is being invalidate because project d773fbf1acca4df9947f053573486931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:51:12.912990 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3411b2bc-8665-4147-805a-5ae301abf225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:12.913666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-3411b2bc-8665-4147-805a-5ae301abf225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:12.915997 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 997/1970] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:51:12 2022] DELETE /identity/v3/projects/d773fbf1acca4df9947f053573486931 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:12.922064 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a4062-4497-46cb-909a-d48afd0e8ed9 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.922555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a4062-4497-46cb-909a-d48afd0e8ed9 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.923074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a4062-4497-46cb-909a-d48afd0e8ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.948954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-405a4062-4497-46cb-909a-d48afd0e8ed9 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:12.950337 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-405a4062-4497-46cb-909a-d48afd0e8ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:13.022957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 974/1971] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:13.031419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a87ce-b403-42e3-903b-acf15d76652f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.031724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a87ce-b403-42e3-903b-acf15d76652f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.031996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a87ce-b403-42e3-903b-acf15d76652f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.052424 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-498a87ce-b403-42e3-903b-acf15d76652f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:13.053644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-498a87ce-b403-42e3-903b-acf15d76652f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:13.062660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 998/1972] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:13.066850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a4062-4497-46cb-909a-d48afd0e8ed9 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.078090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a4062-4497-46cb-909a-d48afd0e8ed9 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.083177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee91d9-9e53-488c-ae89-a68f628238cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.083801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee91d9-9e53-488c-ae89-a68f628238cd admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.084076 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee91d9-9e53-488c-ae89-a68f628238cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.084345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee91d9-9e53-488c-ae89-a68f628238cd admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.085044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee91d9-9e53-488c-ae89-a68f628238cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:13.085347 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee91d9-9e53-488c-ae89-a68f628238cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.087662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee91d9-9e53-488c-ae89-a68f628238cd admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.101700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 975/1973] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:13.106268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a87ce-b403-42e3-903b-acf15d76652f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.116000 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a87ce-b403-42e3-903b-acf15d76652f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.120971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8574da-e6ef-479f-b770-5884939791e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.121594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8574da-e6ef-479f-b770-5884939791e4 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.121904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8574da-e6ef-479f-b770-5884939791e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.122172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8574da-e6ef-479f-b770-5884939791e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.122876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8574da-e6ef-479f-b770-5884939791e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.124778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8574da-e6ef-479f-b770-5884939791e4 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.155951 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 999/1974] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:51:13 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:13.160025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee91d9-9e53-488c-ae89-a68f628238cd admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.170943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee91d9-9e53-488c-ae89-a68f628238cd admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.174580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:13.238355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9056235-cdd4-4040-8a6b-480533673ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.239020 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9056235-cdd4-4040-8a6b-480533673ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.239303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9056235-cdd4-4040-8a6b-480533673ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.239891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9056235-cdd4-4040-8a6b-480533673ab2 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.240799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9056235-cdd4-4040-8a6b-480533673ab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.243062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9056235-cdd4-4040-8a6b-480533673ab2 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.277493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 976/1975] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:51:13 2022] POST /identity/v3/users => generated 303 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:13.282531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8574da-e6ef-479f-b770-5884939791e4 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.295197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8574da-e6ef-479f-b770-5884939791e4 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.299887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a6f9d-1e0a-4643-a718-24e373afd59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.300525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a6f9d-1e0a-4643-a718-24e373afd59d admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.300784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a6f9d-1e0a-4643-a718-24e373afd59d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.301058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a6f9d-1e0a-4643-a718-24e373afd59d admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.301759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a6f9d-1e0a-4643-a718-24e373afd59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.303406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a6f9d-1e0a-4643-a718-24e373afd59d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.318409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1000/1976] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:13.319688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9056235-cdd4-4040-8a6b-480533673ab2 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.331723 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9056235-cdd4-4040-8a6b-480533673ab2 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.336694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3131cf-5aef-455f-a83b-3ef8c9dfa4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.338072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3131cf-5aef-455f-a83b-3ef8c9dfa4de admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.338354 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3131cf-5aef-455f-a83b-3ef8c9dfa4de admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.338662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3131cf-5aef-455f-a83b-3ef8c9dfa4de admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.339370 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3131cf-5aef-455f-a83b-3ef8c9dfa4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.341059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3131cf-5aef-455f-a83b-3ef8c9dfa4de admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.367675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 977/1977] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:13.368164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a6f9d-1e0a-4643-a718-24e373afd59d admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.371188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a6f9d-1e0a-4643-a718-24e373afd59d admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.375845 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b0d2e-e9c0-4424-ad4e-8a51663b515e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.376521 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b0d2e-e9c0-4424-ad4e-8a51663b515e admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.376798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b0d2e-e9c0-4424-ad4e-8a51663b515e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.377067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b0d2e-e9c0-4424-ad4e-8a51663b515e admin admin] PATH_INFO: `/v3/projects/be29a243347148998604ce4c77d8686c/users/9409ab3a60604c80bc0804dc92ac5281/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.394485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b0d2e-e9c0-4424-ad4e-8a51663b515e admin admin] RBAC: Authorizing `identity:create_grant(project_id=be29a243347148998604ce4c77d8686c, user_id=9409ab3a60604c80bc0804dc92ac5281, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.397211 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b0d2e-e9c0-4424-ad4e-8a51663b515e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.418663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1001/1978] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:13 2022] PUT /identity/v3/projects/be29a243347148998604ce4c77d8686c/users/9409ab3a60604c80bc0804dc92ac5281/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:13.429138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4c082-cdf7-4cdc-b9bb-c8d89d7dc5c4 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.429421 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4c082-cdf7-4cdc-b9bb-c8d89d7dc5c4 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.429734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4c082-cdf7-4cdc-b9bb-c8d89d7dc5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.461412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-2be4c082-cdf7-4cdc-b9bb-c8d89d7dc5c4 None None] MFA Rules not processed for user `9409ab3a60604c80bc0804dc92ac5281`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:13.464309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2be4c082-cdf7-4cdc-b9bb-c8d89d7dc5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:13.554279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 978/1979] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:13 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:13.559651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b0d2e-e9c0-4424-ad4e-8a51663b515e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.595200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:13.634973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b0d2e-e9c0-4424-ad4e-8a51663b515e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.642964 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7050a-8609-4aa4-abb3-f0e3e2bae321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.643667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7050a-8609-4aa4-abb3-f0e3e2bae321 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.644010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7050a-8609-4aa4-abb3-f0e3e2bae321 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.644413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7050a-8609-4aa4-abb3-f0e3e2bae321 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.645324 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7050a-8609-4aa4-abb3-f0e3e2bae321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.647375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7050a-8609-4aa4-abb3-f0e3e2bae321 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.686408 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1002/1980] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:51:13 2022] POST /identity/v3/projects => generated 403 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:13.690835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4c082-cdf7-4cdc-b9bb-c8d89d7dc5c4 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.706226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4c082-cdf7-4cdc-b9bb-c8d89d7dc5c4 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.782957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a36be9-f1ac-46c0-8cfe-c550b1d40f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.783599 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a36be9-f1ac-46c0-8cfe-c550b1d40f84 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.783965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a36be9-f1ac-46c0-8cfe-c550b1d40f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.784302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a36be9-f1ac-46c0-8cfe-c550b1d40f84 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.785048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a36be9-f1ac-46c0-8cfe-c550b1d40f84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.788163 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a36be9-f1ac-46c0-8cfe-c550b1d40f84 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.811755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 979/1981] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:51:13 2022] POST /identity/v3/users => generated 308 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:13.815805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7050a-8609-4aa4-abb3-f0e3e2bae321 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.828218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7050a-8609-4aa4-abb3-f0e3e2bae321 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.833726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf980a0-061e-44df-9d1c-d257acca50e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.834336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf980a0-061e-44df-9d1c-d257acca50e1 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.834648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf980a0-061e-44df-9d1c-d257acca50e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.834908 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf980a0-061e-44df-9d1c-d257acca50e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.835837 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf980a0-061e-44df-9d1c-d257acca50e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.837767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf980a0-061e-44df-9d1c-d257acca50e1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.849685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1003/1982] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:13.854092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a36be9-f1ac-46c0-8cfe-c550b1d40f84 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.860697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:13.866004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a36be9-f1ac-46c0-8cfe-c550b1d40f84 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.872578 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3f1c1-a2f0-43c6-9c12-0badf5f5cbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.873249 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f1c1-a2f0-43c6-9c12-0badf5f5cbe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.873528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f1c1-a2f0-43c6-9c12-0badf5f5cbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.873805 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f1c1-a2f0-43c6-9c12-0badf5f5cbe8 admin admin] PATH_INFO: `/v3/projects/fbbe229d9ba74f6389b3b1793264c210/users/6e3607c4fed64b40bccc5357ff6a54a3/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.890422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3f1c1-a2f0-43c6-9c12-0badf5f5cbe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbbe229d9ba74f6389b3b1793264c210, user_id=6e3607c4fed64b40bccc5357ff6a54a3, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.893231 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3f1c1-a2f0-43c6-9c12-0badf5f5cbe8 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.907499 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 980/1983] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:13 2022] PUT /identity/v3/projects/fbbe229d9ba74f6389b3b1793264c210/users/6e3607c4fed64b40bccc5357ff6a54a3/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:13.911523 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf980a0-061e-44df-9d1c-d257acca50e1 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.923085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf980a0-061e-44df-9d1c-d257acca50e1 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.982837 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eba8cf-21f3-4214-a83e-56538c591372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.983458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eba8cf-21f3-4214-a83e-56538c591372 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.983731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eba8cf-21f3-4214-a83e-56538c591372 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.984015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eba8cf-21f3-4214-a83e-56538c591372 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.984687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eba8cf-21f3-4214-a83e-56538c591372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.986271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eba8cf-21f3-4214-a83e-56538c591372 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.998457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1004/1984] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:14.002707 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3f1c1-a2f0-43c6-9c12-0badf5f5cbe8 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:14.015110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3f1c1-a2f0-43c6-9c12-0badf5f5cbe8 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:14.019949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a98993-23a8-4881-8dd5-1a3d511404be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:14.020625 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a98993-23a8-4881-8dd5-1a3d511404be admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:14.020900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a98993-23a8-4881-8dd5-1a3d511404be admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:14.021175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a98993-23a8-4881-8dd5-1a3d511404be admin admin] PATH_INFO: `/v3/projects/fbbe229d9ba74f6389b3b1793264c210/users/6e3607c4fed64b40bccc5357ff6a54a3/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:14.023190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a98993-23a8-4881-8dd5-1a3d511404be admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbbe229d9ba74f6389b3b1793264c210, user_id=6e3607c4fed64b40bccc5357ff6a54a3, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:14.026187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a98993-23a8-4881-8dd5-1a3d511404be admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:14.037691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 981/1985] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:14 2022] PUT /identity/v3/projects/fbbe229d9ba74f6389b3b1793264c210/users/6e3607c4fed64b40bccc5357ff6a54a3/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:14.050439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac45209-d8f9-4ba2-9685-dd9f55fcd60c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:14.050951 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac45209-d8f9-4ba2-9685-dd9f55fcd60c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:14.051330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac45209-d8f9-4ba2-9685-dd9f55fcd60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:14.082653 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-bac45209-d8f9-4ba2-9685-dd9f55fcd60c None None] MFA Rules not processed for user `6e3607c4fed64b40bccc5357ff6a54a3`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:14.083950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-bac45209-d8f9-4ba2-9685-dd9f55fcd60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:14.161469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1005/1986] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:14 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:14.174573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a98993-23a8-4881-8dd5-1a3d511404be admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:14.176975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-82a98993-23a8-4881-8dd5-1a3d511404be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:14.240629 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a98993-23a8-4881-8dd5-1a3d511404be admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:14.245324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb29efb-8b6f-4da3-ba16-3407ce6e6bf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:14.245964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb29efb-8b6f-4da3-ba16-3407ce6e6bf4 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:14.246242 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb29efb-8b6f-4da3-ba16-3407ce6e6bf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:14.246508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb29efb-8b6f-4da3-ba16-3407ce6e6bf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:14.259052 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb29efb-8b6f-4da3-ba16-3407ce6e6bf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:14.260504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb29efb-8b6f-4da3-ba16-3407ce6e6bf4 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:14.270965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 982/1987] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:14 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:51:14.567781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac45209-d8f9-4ba2-9685-dd9f55fcd60c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:14.596160 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:14.639807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac45209-d8f9-4ba2-9685-dd9f55fcd60c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:14.644657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:14.645477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:14.645782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:14.646028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] PATH_INFO: `/v3/users/9409ab3a60604c80bc0804dc92ac5281` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:14.647254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9409ab3a60604c80bc0804dc92ac5281)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:14.649841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:14.700179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9409ab3a60604c80bc0804dc92ac5281'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:14.713724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9409ab3a60604c80bc0804dc92ac5281'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:14.722416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9409ab3a60604c80bc0804dc92ac5281'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:14.723190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9409ab3a60604c80bc0804dc92ac5281'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:14.723819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9409ab3a60604c80bc0804dc92ac5281'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:14.739480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1006/1988] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:51:14 2022] DELETE /identity/v3/users/9409ab3a60604c80bc0804dc92ac5281 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:14.744734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd21d98-aeda-4318-8473-94268097b496 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:14.745205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd21d98-aeda-4318-8473-94268097b496 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:14.745306 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd21d98-aeda-4318-8473-94268097b496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:14.769345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-afd21d98-aeda-4318-8473-94268097b496 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:14.770120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-afd21d98-aeda-4318-8473-94268097b496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:14.842233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 983/1989] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:14.854008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:14.856261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:14.927166 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445dc16-01df-4906-8102-a624e8a4045b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:14.932712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdca413-3efa-4db2-bfad-b37a41e29af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:14.933388 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdca413-3efa-4db2-bfad-b37a41e29af8 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:14.933725 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdca413-3efa-4db2-bfad-b37a41e29af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:14.933991 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdca413-3efa-4db2-bfad-b37a41e29af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:14.944808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdca413-3efa-4db2-bfad-b37a41e29af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:14.946809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdca413-3efa-4db2-bfad-b37a41e29af8 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:14.959893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1007/1990] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:51:15.272842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd21d98-aeda-4318-8473-94268097b496 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:15.275759 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-afd21d98-aeda-4318-8473-94268097b496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:15.288918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd21d98-aeda-4318-8473-94268097b496 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:15.294053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:15.294876 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:15.295192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:15.295436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] PATH_INFO: `/v3/projects/be29a243347148998604ce4c77d8686c` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:15.297014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be29a243347148998604ce4c77d8686c)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:15.300718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:15.375091 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be29a243347148998604ce4c77d8686c', 'request_id': 'req-71c3cfac-c21f-4c31-93ee-0d574c01bce1'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:15.387427 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be29a243347148998604ce4c77d8686c', 'request_id': 'req-71c3cfac-c21f-4c31-93ee-0d574c01bce1'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:15.400096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be29a243347148998604ce4c77d8686c', 'request_id': 'req-71c3cfac-c21f-4c31-93ee-0d574c01bce1'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:15.417819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:15.422513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] The token cache is being invalidate because project be29a243347148998604ce4c77d8686c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:51:15.422953 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:15.423677 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:15.425346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 984/1991] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:51:15 2022] DELETE /identity/v3/projects/be29a243347148998604ce4c77d8686c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:15.429351 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdca413-3efa-4db2-bfad-b37a41e29af8 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:15.440191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdca413-3efa-4db2-bfad-b37a41e29af8 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:15.484642 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:15.501641 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101361c-449c-4c90-80aa-b6aff1135481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:15.502290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101361c-449c-4c90-80aa-b6aff1135481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:15.502569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101361c-449c-4c90-80aa-b6aff1135481 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:15.502886 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101361c-449c-4c90-80aa-b6aff1135481 admin admin] PATH_INFO: `/v3/users/6e3607c4fed64b40bccc5357ff6a54a3` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:15.504075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101361c-449c-4c90-80aa-b6aff1135481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e3607c4fed64b40bccc5357ff6a54a3)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:15.506010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101361c-449c-4c90-80aa-b6aff1135481 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:15.555140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b101361c-449c-4c90-80aa-b6aff1135481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e3607c4fed64b40bccc5357ff6a54a3'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:15.572151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b101361c-449c-4c90-80aa-b6aff1135481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e3607c4fed64b40bccc5357ff6a54a3'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:15.582162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b101361c-449c-4c90-80aa-b6aff1135481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e3607c4fed64b40bccc5357ff6a54a3'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:15.582968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b101361c-449c-4c90-80aa-b6aff1135481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e3607c4fed64b40bccc5357ff6a54a3'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:15.583606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b101361c-449c-4c90-80aa-b6aff1135481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e3607c4fed64b40bccc5357ff6a54a3'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:15.596433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1008/1992] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:51:15 2022] DELETE /identity/v3/users/6e3607c4fed64b40bccc5357ff6a54a3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:15.892951 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:15.895241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:15.910449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c3cfac-c21f-4c31-93ee-0d574c01bce1 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:15.975884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:15.977146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:15.977495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:15.977760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628 admin admin] PATH_INFO: `/v3/projects/fbbe229d9ba74f6389b3b1793264c210` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:15.979227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbbe229d9ba74f6389b3b1793264c210)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:15.981142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.032435 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbbe229d9ba74f6389b3b1793264c210', 'request_id': 'req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:16.039912 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbbe229d9ba74f6389b3b1793264c210', 'request_id': 'req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:16.048343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbbe229d9ba74f6389b3b1793264c210', 'request_id': 'req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:16.066350 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628 admin admin] The token cache is being invalidate because project fbbe229d9ba74f6389b3b1793264c210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:51:16.066814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:16.067759 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5a6b1e4d-d862-4aef-a1b8-a2a62a332628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:16.070228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 985/1993] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:51:15 2022] DELETE /identity/v3/projects/fbbe229d9ba74f6389b3b1793264c210 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:16.077823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbf92a-8e82-47b4-b6ba-e70f7795c14b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.078454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbf92a-8e82-47b4-b6ba-e70f7795c14b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.078454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbf92a-8e82-47b4-b6ba-e70f7795c14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.099494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c0dbf92a-8e82-47b4-b6ba-e70f7795c14b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:16.100716 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c0dbf92a-8e82-47b4-b6ba-e70f7795c14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:16.162934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1009/1994] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:51:16.170917 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ff136-f6f2-49b3-99bf-1d29da3d2b2c None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.171288 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ff136-f6f2-49b3-99bf-1d29da3d2b2c None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.171542 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ff136-f6f2-49b3-99bf-1d29da3d2b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.192839 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-b88ff136-f6f2-49b3-99bf-1d29da3d2b2c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:16.194055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b88ff136-f6f2-49b3-99bf-1d29da3d2b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:16.203098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 986/1995] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:16.207516 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbf92a-8e82-47b4-b6ba-e70f7795c14b None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.219409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbf92a-8e82-47b4-b6ba-e70f7795c14b None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.224135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bcf99-fc91-49bd-a652-58f66e49cea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.225086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bcf99-fc91-49bd-a652-58f66e49cea7 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.225086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bcf99-fc91-49bd-a652-58f66e49cea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.225321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bcf99-fc91-49bd-a652-58f66e49cea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.225962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bcf99-fc91-49bd-a652-58f66e49cea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:16.226324 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bcf99-fc91-49bd-a652-58f66e49cea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.228242 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bcf99-fc91-49bd-a652-58f66e49cea7 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.246219 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1010/1996] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:16.247753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ff136-f6f2-49b3-99bf-1d29da3d2b2c None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.258225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ff136-f6f2-49b3-99bf-1d29da3d2b2c None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.264799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152da43f-7bad-4b56-9901-70ec110b3b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.265421 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152da43f-7bad-4b56-9901-70ec110b3b02 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.265697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152da43f-7bad-4b56-9901-70ec110b3b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.265969 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152da43f-7bad-4b56-9901-70ec110b3b02 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.266655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152da43f-7bad-4b56-9901-70ec110b3b02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.268561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152da43f-7bad-4b56-9901-70ec110b3b02 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.306493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 987/1997] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:51:16 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:16.307996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bcf99-fc91-49bd-a652-58f66e49cea7 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.318228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bcf99-fc91-49bd-a652-58f66e49cea7 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.369784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:16.384590 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf409e0-a302-4ad9-956b-6508fd129920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.385222 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf409e0-a302-4ad9-956b-6508fd129920 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.385500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf409e0-a302-4ad9-956b-6508fd129920 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.385775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf409e0-a302-4ad9-956b-6508fd129920 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.386451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf409e0-a302-4ad9-956b-6508fd129920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.388425 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf409e0-a302-4ad9-956b-6508fd129920 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.410451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1011/1998] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:51:16 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:16.414676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152da43f-7bad-4b56-9901-70ec110b3b02 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.428269 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152da43f-7bad-4b56-9901-70ec110b3b02 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.435012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ba3e7-a15c-4782-91d2-83446cbe8b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.435827 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ba3e7-a15c-4782-91d2-83446cbe8b26 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.436176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ba3e7-a15c-4782-91d2-83446cbe8b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.436553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ba3e7-a15c-4782-91d2-83446cbe8b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.437218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ba3e7-a15c-4782-91d2-83446cbe8b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.438936 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ba3e7-a15c-4782-91d2-83446cbe8b26 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.453868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 988/1999] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:16.455147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf409e0-a302-4ad9-956b-6508fd129920 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.465883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf409e0-a302-4ad9-956b-6508fd129920 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.470887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9293dd4-bf9d-4a0c-8c7e-c21edd7fd0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.471508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9293dd4-bf9d-4a0c-8c7e-c21edd7fd0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.471790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9293dd4-bf9d-4a0c-8c7e-c21edd7fd0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.472063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9293dd4-bf9d-4a0c-8c7e-c21edd7fd0d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.472807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9293dd4-bf9d-4a0c-8c7e-c21edd7fd0d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.474408 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9293dd4-bf9d-4a0c-8c7e-c21edd7fd0d5 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.487949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1012/2000] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:16.492163 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ba3e7-a15c-4782-91d2-83446cbe8b26 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.505223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ba3e7-a15c-4782-91d2-83446cbe8b26 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.510341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bdd68-6e92-4276-9014-1c25d000eb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.511063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bdd68-6e92-4276-9014-1c25d000eb87 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.511341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bdd68-6e92-4276-9014-1c25d000eb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.511619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bdd68-6e92-4276-9014-1c25d000eb87 admin admin] PATH_INFO: `/v3/projects/ba711a7c6df74c6da770803cc2ef8d6f/users/8b7e9108ed964ab1b4b3e900831ea31b/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.528063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bdd68-6e92-4276-9014-1c25d000eb87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba711a7c6df74c6da770803cc2ef8d6f, user_id=8b7e9108ed964ab1b4b3e900831ea31b, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.530744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bdd68-6e92-4276-9014-1c25d000eb87 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.548650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 989/2001] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:16 2022] PUT /identity/v3/projects/ba711a7c6df74c6da770803cc2ef8d6f/users/8b7e9108ed964ab1b4b3e900831ea31b/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:16.560324 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b0441-6e6a-4eb0-82ed-e17e5fa87e48 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.560608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b0441-6e6a-4eb0-82ed-e17e5fa87e48 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.560880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b0441-6e6a-4eb0-82ed-e17e5fa87e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.592404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-0b6b0441-6e6a-4eb0-82ed-e17e5fa87e48 None None] MFA Rules not processed for user `8b7e9108ed964ab1b4b3e900831ea31b`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:16.593689 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-0b6b0441-6e6a-4eb0-82ed-e17e5fa87e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:16.673928 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1013/2002] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:16.678696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bdd68-6e92-4276-9014-1c25d000eb87 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.711502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:16.757888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bdd68-6e92-4276-9014-1c25d000eb87 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.764090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748bf04-5e8f-4bb0-a6fd-f20d645d6e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.764708 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748bf04-5e8f-4bb0-a6fd-f20d645d6e47 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.764982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748bf04-5e8f-4bb0-a6fd-f20d645d6e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.765252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748bf04-5e8f-4bb0-a6fd-f20d645d6e47 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.765903 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748bf04-5e8f-4bb0-a6fd-f20d645d6e47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.767829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748bf04-5e8f-4bb0-a6fd-f20d645d6e47 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.797695 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 990/2003] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:51:16 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:16.801579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b0441-6e6a-4eb0-82ed-e17e5fa87e48 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.813582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b0441-6e6a-4eb0-82ed-e17e5fa87e48 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.879208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0f85d-f6e3-4506-a9dd-6f3e9b7a6910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.879977 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0f85d-f6e3-4506-a9dd-6f3e9b7a6910 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.880251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0f85d-f6e3-4506-a9dd-6f3e9b7a6910 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.880516 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0f85d-f6e3-4506-a9dd-6f3e9b7a6910 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.881176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0f85d-f6e3-4506-a9dd-6f3e9b7a6910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.883146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0f85d-f6e3-4506-a9dd-6f3e9b7a6910 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.908195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1014/2004] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:51:16 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:16.909323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748bf04-5e8f-4bb0-a6fd-f20d645d6e47 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.921612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748bf04-5e8f-4bb0-a6fd-f20d645d6e47 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.926378 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451ca91-80a0-4a33-a2a9-3d3383cdfd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.926957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451ca91-80a0-4a33-a2a9-3d3383cdfd0d admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.927235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451ca91-80a0-4a33-a2a9-3d3383cdfd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.927503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451ca91-80a0-4a33-a2a9-3d3383cdfd0d admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.928196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451ca91-80a0-4a33-a2a9-3d3383cdfd0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.929802 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451ca91-80a0-4a33-a2a9-3d3383cdfd0d admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.941949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 991/2005] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:51:16.946479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0f85d-f6e3-4506-a9dd-6f3e9b7a6910 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.956902 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0f85d-f6e3-4506-a9dd-6f3e9b7a6910 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.961450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50923c-9044-4bd7-84fa-a6140549676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.963024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50923c-9044-4bd7-84fa-a6140549676a admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.963447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50923c-9044-4bd7-84fa-a6140549676a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.963730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50923c-9044-4bd7-84fa-a6140549676a admin admin] PATH_INFO: `/v3/projects/a6eae51ada44491e851e1c80f953e0eb/users/ff5a4265507644dfa4d9ccaed1ab5d83/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.979934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50923c-9044-4bd7-84fa-a6140549676a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6eae51ada44491e851e1c80f953e0eb, user_id=ff5a4265507644dfa4d9ccaed1ab5d83, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.982646 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50923c-9044-4bd7-84fa-a6140549676a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.998218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1015/2006] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:16 2022] PUT /identity/v3/projects/a6eae51ada44491e851e1c80f953e0eb/users/ff5a4265507644dfa4d9ccaed1ab5d83/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:17.002364 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451ca91-80a0-4a33-a2a9-3d3383cdfd0d admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:17.012946 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451ca91-80a0-4a33-a2a9-3d3383cdfd0d admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:17.073590 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803ad7f-681d-46b7-9aaf-d910ad2f9e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:17.074214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803ad7f-681d-46b7-9aaf-d910ad2f9e47 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:17.074497 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803ad7f-681d-46b7-9aaf-d910ad2f9e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:17.074796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803ad7f-681d-46b7-9aaf-d910ad2f9e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:17.075486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803ad7f-681d-46b7-9aaf-d910ad2f9e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:17.077114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803ad7f-681d-46b7-9aaf-d910ad2f9e47 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:17.088756 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 992/2007] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:17.092989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50923c-9044-4bd7-84fa-a6140549676a admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:17.104763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50923c-9044-4bd7-84fa-a6140549676a admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:17.110270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a43a7-59ee-472d-a5b4-f5c6d74ba9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:17.111047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a43a7-59ee-472d-a5b4-f5c6d74ba9f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:17.111322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a43a7-59ee-472d-a5b4-f5c6d74ba9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:17.111597 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a43a7-59ee-472d-a5b4-f5c6d74ba9f1 admin admin] PATH_INFO: `/v3/projects/a6eae51ada44491e851e1c80f953e0eb/users/ff5a4265507644dfa4d9ccaed1ab5d83/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:17.113575 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a43a7-59ee-472d-a5b4-f5c6d74ba9f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6eae51ada44491e851e1c80f953e0eb, user_id=ff5a4265507644dfa4d9ccaed1ab5d83, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:17.116271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a43a7-59ee-472d-a5b4-f5c6d74ba9f1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:17.128237 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1016/2008] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:17 2022] PUT /identity/v3/projects/a6eae51ada44491e851e1c80f953e0eb/users/ff5a4265507644dfa4d9ccaed1ab5d83/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:51:17.140764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5392ec-21e9-42bc-9eff-5ad6420aa30e None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:17.141227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5392ec-21e9-42bc-9eff-5ad6420aa30e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:17.141555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5392ec-21e9-42bc-9eff-5ad6420aa30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:17.185099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-2a5392ec-21e9-42bc-9eff-5ad6420aa30e None None] MFA Rules not processed for user `ff5a4265507644dfa4d9ccaed1ab5d83`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:17.186657 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2a5392ec-21e9-42bc-9eff-5ad6420aa30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:17.263757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 993/2009] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:17.280873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a43a7-59ee-472d-a5b4-f5c6d74ba9f1 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:17.282750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ab2a43a7-59ee-472d-a5b4-f5c6d74ba9f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:17.300289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:17.351937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a43a7-59ee-472d-a5b4-f5c6d74ba9f1 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:17.357622 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee701bb5-4099-4646-b251-eb322b4181d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:17.358370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee701bb5-4099-4646-b251-eb322b4181d3 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:17.368138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee701bb5-4099-4646-b251-eb322b4181d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:17.368138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee701bb5-4099-4646-b251-eb322b4181d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:17.421582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee701bb5-4099-4646-b251-eb322b4181d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:17.423175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee701bb5-4099-4646-b251-eb322b4181d3 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:17.434228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1017/2010] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:51:18.468394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5392ec-21e9-42bc-9eff-5ad6420aa30e None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:18.469815 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2a5392ec-21e9-42bc-9eff-5ad6420aa30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:18.495739 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:18.545559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5392ec-21e9-42bc-9eff-5ad6420aa30e None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:18.550118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:18.550798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:18.551076 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:18.551344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] PATH_INFO: `/v3/users/fb082a67f98d4b12bcf5c8f97fff2cfd` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:18.552505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb082a67f98d4b12bcf5c8f97fff2cfd)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:18.554414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:18.603592 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb082a67f98d4b12bcf5c8f97fff2cfd'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:18.610586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb082a67f98d4b12bcf5c8f97fff2cfd'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:18.611308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb082a67f98d4b12bcf5c8f97fff2cfd'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:18.624872 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb082a67f98d4b12bcf5c8f97fff2cfd'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:18.633251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb082a67f98d4b12bcf5c8f97fff2cfd'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:18.641177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 994/2011] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:51:18 2022] DELETE /identity/v3/users/fb082a67f98d4b12bcf5c8f97fff2cfd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:18.645824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd083a6-5852-42e0-b246-300086bba0c1 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:18.646310 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd083a6-5852-42e0-b246-300086bba0c1 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:18.646429 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd083a6-5852-42e0-b246-300086bba0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:18.676267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3bd083a6-5852-42e0-b246-300086bba0c1 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:18.677471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3bd083a6-5852-42e0-b246-300086bba0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:18.743960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1018/2012] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:18.758642 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:18.760203 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:18.820605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441e036-7d8e-4e12-b78b-01b7c0d8a98a admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:18.824778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2eda8-9bda-406e-9ab5-8d0755f464be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:18.826696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2eda8-9bda-406e-9ab5-8d0755f464be service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:18.826696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2eda8-9bda-406e-9ab5-8d0755f464be service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:18.826696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2eda8-9bda-406e-9ab5-8d0755f464be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:18.843214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2eda8-9bda-406e-9ab5-8d0755f464be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:18.845442 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2eda8-9bda-406e-9ab5-8d0755f464be service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:18.860746 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 995/2013] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:51:19.263156 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd083a6-5852-42e0-b246-300086bba0c1 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:19.264622 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3bd083a6-5852-42e0-b246-300086bba0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:19.277444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd083a6-5852-42e0-b246-300086bba0c1 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:19.282763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f05df-8eca-49b8-9933-a46a06ce9603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:19.283430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f05df-8eca-49b8-9933-a46a06ce9603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.283711 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f05df-8eca-49b8-9933-a46a06ce9603 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.283987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f05df-8eca-49b8-9933-a46a06ce9603 admin admin] PATH_INFO: `/v3/projects/6bced703b1fa483492888f907d4b5c68` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.285228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f05df-8eca-49b8-9933-a46a06ce9603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bced703b1fa483492888f907d4b5c68)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:19.287322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f05df-8eca-49b8-9933-a46a06ce9603 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:19.340703 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-170f05df-8eca-49b8-9933-a46a06ce9603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bced703b1fa483492888f907d4b5c68', 'request_id': 'req-170f05df-8eca-49b8-9933-a46a06ce9603'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:19.347038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-170f05df-8eca-49b8-9933-a46a06ce9603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bced703b1fa483492888f907d4b5c68', 'request_id': 'req-170f05df-8eca-49b8-9933-a46a06ce9603'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:19.356226 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-170f05df-8eca-49b8-9933-a46a06ce9603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bced703b1fa483492888f907d4b5c68', 'request_id': 'req-170f05df-8eca-49b8-9933-a46a06ce9603'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:19.357680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:19.370976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-170f05df-8eca-49b8-9933-a46a06ce9603 admin admin] The token cache is being invalidate because project 6bced703b1fa483492888f907d4b5c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:51:19.370976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-170f05df-8eca-49b8-9933-a46a06ce9603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:19.370976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-170f05df-8eca-49b8-9933-a46a06ce9603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:19.371957 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1019/2014] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:51:19 2022] DELETE /identity/v3/projects/6bced703b1fa483492888f907d4b5c68 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:19.377402 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac564cbd-1ef0-4309-8ce9-2ad6fac424d2 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.377694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac564cbd-1ef0-4309-8ce9-2ad6fac424d2 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.378004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac564cbd-1ef0-4309-8ce9-2ad6fac424d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.401797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-ac564cbd-1ef0-4309-8ce9-2ad6fac424d2 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:19.403082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ac564cbd-1ef0-4309-8ce9-2ad6fac424d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:19.442695 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:19.479815 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 996/2015] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:19.488220 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843d5e8-9f0d-4cc6-84eb-97964ac7eb55 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.488506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843d5e8-9f0d-4cc6-84eb-97964ac7eb55 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.488931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843d5e8-9f0d-4cc6-84eb-97964ac7eb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.514045 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3843d5e8-9f0d-4cc6-84eb-97964ac7eb55 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:19.514045 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3843d5e8-9f0d-4cc6-84eb-97964ac7eb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:19.520844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1020/2016] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:19.525424 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac564cbd-1ef0-4309-8ce9-2ad6fac424d2 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:19.535791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac564cbd-1ef0-4309-8ce9-2ad6fac424d2 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:19.540699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77bd76-b5dd-4547-b12f-db549cf34e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:19.541312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77bd76-b5dd-4547-b12f-db549cf34e6f admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.541586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77bd76-b5dd-4547-b12f-db549cf34e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.541852 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77bd76-b5dd-4547-b12f-db549cf34e6f admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.542545 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77bd76-b5dd-4547-b12f-db549cf34e6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:19.542891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77bd76-b5dd-4547-b12f-db549cf34e6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:19.544516 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77bd76-b5dd-4547-b12f-db549cf34e6f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:19.558231 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 997/2017] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:19.562906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843d5e8-9f0d-4cc6-84eb-97964ac7eb55 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:19.573298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843d5e8-9f0d-4cc6-84eb-97964ac7eb55 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:19.578079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751231ae-b771-4e6f-90ea-0058e87bc346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:19.578734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751231ae-b771-4e6f-90ea-0058e87bc346 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.579013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751231ae-b771-4e6f-90ea-0058e87bc346 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.579284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751231ae-b771-4e6f-90ea-0058e87bc346 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.579959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751231ae-b771-4e6f-90ea-0058e87bc346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:19.581858 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751231ae-b771-4e6f-90ea-0058e87bc346 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:19.615794 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1021/2018] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:51:19 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:19.616867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77bd76-b5dd-4547-b12f-db549cf34e6f admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:19.628228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77bd76-b5dd-4547-b12f-db549cf34e6f admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:19.694742 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5a7fa-d42b-409c-98e1-7ad4c227b2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:19.695360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5a7fa-d42b-409c-98e1-7ad4c227b2f0 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.695640 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5a7fa-d42b-409c-98e1-7ad4c227b2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.696957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5a7fa-d42b-409c-98e1-7ad4c227b2f0 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.698145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5a7fa-d42b-409c-98e1-7ad4c227b2f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:19.700462 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5a7fa-d42b-409c-98e1-7ad4c227b2f0 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:19.729039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 998/2019] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:51:19 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:51:19.734252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751231ae-b771-4e6f-90ea-0058e87bc346 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:19.749146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751231ae-b771-4e6f-90ea-0058e87bc346 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:19.753903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85827a-e289-459c-b1a0-df36a56f0b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:19.754518 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85827a-e289-459c-b1a0-df36a56f0b80 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.754821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85827a-e289-459c-b1a0-df36a56f0b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.755087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85827a-e289-459c-b1a0-df36a56f0b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.755779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85827a-e289-459c-b1a0-df36a56f0b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:19.757367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85827a-e289-459c-b1a0-df36a56f0b80 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:19.771805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1022/2020] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:19.775987 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5a7fa-d42b-409c-98e1-7ad4c227b2f0 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:19.786657 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5a7fa-d42b-409c-98e1-7ad4c227b2f0 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:19.792059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44515d-5947-499e-9e45-fa473727a56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:19.792675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44515d-5947-499e-9e45-fa473727a56c admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.792952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44515d-5947-499e-9e45-fa473727a56c admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.793221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44515d-5947-499e-9e45-fa473727a56c admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.793918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44515d-5947-499e-9e45-fa473727a56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:19.795830 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44515d-5947-499e-9e45-fa473727a56c admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:19.809051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 999/2021] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:19.813369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85827a-e289-459c-b1a0-df36a56f0b80 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:19.823862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85827a-e289-459c-b1a0-df36a56f0b80 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:19.833536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6280d55-e920-483a-ab4b-0f8bb1286ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:19.834219 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6280d55-e920-483a-ab4b-0f8bb1286ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.834496 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6280d55-e920-483a-ab4b-0f8bb1286ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.834804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6280d55-e920-483a-ab4b-0f8bb1286ef0 admin admin] PATH_INFO: `/v3/projects/61e9cc35b8d242cf9165158977a3ccbd/users/e395cb0b011447ca8b4410d812239133/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.850288 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6280d55-e920-483a-ab4b-0f8bb1286ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e9cc35b8d242cf9165158977a3ccbd, user_id=e395cb0b011447ca8b4410d812239133, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:19.853514 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6280d55-e920-483a-ab4b-0f8bb1286ef0 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:19.867623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1023/2022] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:19 2022] PUT /identity/v3/projects/61e9cc35b8d242cf9165158977a3ccbd/users/e395cb0b011447ca8b4410d812239133/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:19.880680 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1170-71ba-4320-9c76-ee007a8fe2d7 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.881140 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1170-71ba-4320-9c76-ee007a8fe2d7 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.881453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1170-71ba-4320-9c76-ee007a8fe2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.913229 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-7c9e1170-71ba-4320-9c76-ee007a8fe2d7 None None] MFA Rules not processed for user `e395cb0b011447ca8b4410d812239133`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:19.914472 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7c9e1170-71ba-4320-9c76-ee007a8fe2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:19.983385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1000/2023] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:19.995469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6280d55-e920-483a-ab4b-0f8bb1286ef0 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:19.997321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d6280d55-e920-483a-ab4b-0f8bb1286ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:20.064267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6280d55-e920-483a-ab4b-0f8bb1286ef0 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:20.068137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067ca1b-3326-43c1-9ec8-d0fa0c2df512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:20.068766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067ca1b-3326-43c1-9ec8-d0fa0c2df512 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:20.069046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067ca1b-3326-43c1-9ec8-d0fa0c2df512 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:20.069348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067ca1b-3326-43c1-9ec8-d0fa0c2df512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:20.079646 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067ca1b-3326-43c1-9ec8-d0fa0c2df512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:20.081087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067ca1b-3326-43c1-9ec8-d0fa0c2df512 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:20.093084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1024/2024] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:51:23.214406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e1170-71ba-4320-9c76-ee007a8fe2d7 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:23.215941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7c9e1170-71ba-4320-9c76-ee007a8fe2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:23.231385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e1170-71ba-4320-9c76-ee007a8fe2d7 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:23.234605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0db39-9719-48d9-8a72-f6f836330ed3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:23.235250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0db39-9719-48d9-8a72-f6f836330ed3 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:23.235527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0db39-9719-48d9-8a72-f6f836330ed3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:23.235797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0db39-9719-48d9-8a72-f6f836330ed3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:23.237588 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a2a0db39-9719-48d9-8a72-f6f836330ed3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:23.273226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:23.309551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0db39-9719-48d9-8a72-f6f836330ed3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:23.311021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0db39-9719-48d9-8a72-f6f836330ed3 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:23.321059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1001/2025] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:51:27.826375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067ca1b-3326-43c1-9ec8-d0fa0c2df512 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:27.827850 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2067ca1b-3326-43c1-9ec8-d0fa0c2df512 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:27.913400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067ca1b-3326-43c1-9ec8-d0fa0c2df512 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:27.919767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:27.920431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:27.920702 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:27.920976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] PATH_INFO: `/v3/users/79a114faf69241c585dcaf3ec3c1c066` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:27.922162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a114faf69241c585dcaf3ec3c1c066)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:27.924151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:27.969363 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:27.987070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a114faf69241c585dcaf3ec3c1c066'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.000005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a114faf69241c585dcaf3ec3c1c066'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.008116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a114faf69241c585dcaf3ec3c1c066'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.010561 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a114faf69241c585dcaf3ec3c1c066'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.010561 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a114faf69241c585dcaf3ec3c1c066'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.023352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1025/2026] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:51:27 2022] DELETE /identity/v3/users/79a114faf69241c585dcaf3ec3c1c066 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:28.028354 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb0621-ba11-4692-bc4d-b893bf477fd3 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:28.028882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb0621-ba11-4692-bc4d-b893bf477fd3 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:28.029045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb0621-ba11-4692-bc4d-b893bf477fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:28.048929 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-05cb0621-ba11-4692-bc4d-b893bf477fd3 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:28.050190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-05cb0621-ba11-4692-bc4d-b893bf477fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:28.112325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1002/2027] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:28.123960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:28.125483 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:28.190804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5034ed-0772-4404-8803-abc390b9fe32 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:28.196985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9f280-1d7e-4506-87a5-d0a57d3e750e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:28.197614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9f280-1d7e-4506-87a5-d0a57d3e750e service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:28.197887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9f280-1d7e-4506-87a5-d0a57d3e750e service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:28.198160 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9f280-1d7e-4506-87a5-d0a57d3e750e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:28.207877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9f280-1d7e-4506-87a5-d0a57d3e750e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:28.209334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9f280-1d7e-4506-87a5-d0a57d3e750e service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:28.221668 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1026/2028] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:51:28.740774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb0621-ba11-4692-bc4d-b893bf477fd3 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:28.743570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-05cb0621-ba11-4692-bc4d-b893bf477fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:28.757551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb0621-ba11-4692-bc4d-b893bf477fd3 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:28.765939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fc3b7-0433-4f31-bde6-8d8e34ecbd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:28.766596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fc3b7-0433-4f31-bde6-8d8e34ecbd18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:28.766904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fc3b7-0433-4f31-bde6-8d8e34ecbd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:28.767170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fc3b7-0433-4f31-bde6-8d8e34ecbd18 admin admin] PATH_INFO: `/v3/users/e395cb0b011447ca8b4410d812239133` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:28.768353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4fc3b7-0433-4f31-bde6-8d8e34ecbd18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e395cb0b011447ca8b4410d812239133)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:28.770287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4fc3b7-0433-4f31-bde6-8d8e34ecbd18 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:28.814573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9f280-1d7e-4506-87a5-d0a57d3e750e service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:28.816277 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-eab9f280-1d7e-4506-87a5-d0a57d3e750e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:28.830443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9f280-1d7e-4506-87a5-d0a57d3e750e service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:28.841581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a9882-34a7-4d3e-800a-09a681be3ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:28.842160 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a9882-34a7-4d3e-800a-09a681be3ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:28.843011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a9882-34a7-4d3e-800a-09a681be3ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:28.843300 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a9882-34a7-4d3e-800a-09a681be3ecc admin admin] PATH_INFO: `/v3/projects/02c6b82d3ad74e0caa781f6184b67c42` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:28.844511 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a9882-34a7-4d3e-800a-09a681be3ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c6b82d3ad74e0caa781f6184b67c42)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:28.846415 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a9882-34a7-4d3e-800a-09a681be3ecc admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:28.866689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-aa4fc3b7-0433-4f31-bde6-8d8e34ecbd18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e395cb0b011447ca8b4410d812239133'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.877174 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-aa4fc3b7-0433-4f31-bde6-8d8e34ecbd18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e395cb0b011447ca8b4410d812239133'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.878182 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-aa4fc3b7-0433-4f31-bde6-8d8e34ecbd18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e395cb0b011447ca8b4410d812239133'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.894835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-aa4fc3b7-0433-4f31-bde6-8d8e34ecbd18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e395cb0b011447ca8b4410d812239133'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.905315 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-aa4fc3b7-0433-4f31-bde6-8d8e34ecbd18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e395cb0b011447ca8b4410d812239133'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.912619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1003/2029] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:51:28 2022] DELETE /identity/v3/users/e395cb0b011447ca8b4410d812239133 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:28.913460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-321a9882-34a7-4d3e-800a-09a681be3ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c6b82d3ad74e0caa781f6184b67c42', 'request_id': 'req-321a9882-34a7-4d3e-800a-09a681be3ecc'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.917613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc18bac-928f-4e02-baeb-d2531ecd0701 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:28.917895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc18bac-928f-4e02-baeb-d2531ecd0701 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:28.918169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc18bac-928f-4e02-baeb-d2531ecd0701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:28.921267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-321a9882-34a7-4d3e-800a-09a681be3ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c6b82d3ad74e0caa781f6184b67c42', 'request_id': 'req-321a9882-34a7-4d3e-800a-09a681be3ecc'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.933933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-321a9882-34a7-4d3e-800a-09a681be3ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c6b82d3ad74e0caa781f6184b67c42', 'request_id': 'req-321a9882-34a7-4d3e-800a-09a681be3ecc'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.940601 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-7cc18bac-928f-4e02-baeb-d2531ecd0701 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:28.941823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7cc18bac-928f-4e02-baeb-d2531ecd0701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:28.945427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-321a9882-34a7-4d3e-800a-09a681be3ecc admin admin] The token cache is being invalidate because project 02c6b82d3ad74e0caa781f6184b67c42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:51:28.945937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-321a9882-34a7-4d3e-800a-09a681be3ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.946463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-321a9882-34a7-4d3e-800a-09a681be3ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:28.948049 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1027/2030] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:51:28 2022] DELETE /identity/v3/projects/02c6b82d3ad74e0caa781f6184b67c42 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:28.952965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dafe3-552c-4844-8914-e44086db3f53 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:28.953283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dafe3-552c-4844-8914-e44086db3f53 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:28.953562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dafe3-552c-4844-8914-e44086db3f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:28.975180 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-788dafe3-552c-4844-8914-e44086db3f53 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:28.976639 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-788dafe3-552c-4844-8914-e44086db3f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:28.982977 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:29.011019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:29.015731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1004/2031] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:51:29.025461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc18bac-928f-4e02-baeb-d2531ecd0701 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:29.027169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7cc18bac-928f-4e02-baeb-d2531ecd0701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:29.056229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1028/2032] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:29.066407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d846d5-e161-4b1d-aa86-f06d2f8ba235 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.066829 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d846d5-e161-4b1d-aa86-f06d2f8ba235 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.067102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d846d5-e161-4b1d-aa86-f06d2f8ba235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.098353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-63d846d5-e161-4b1d-aa86-f06d2f8ba235 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:29.099781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc18bac-928f-4e02-baeb-d2531ecd0701 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:29.100768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-63d846d5-e161-4b1d-aa86-f06d2f8ba235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:29.104166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d93031-02e4-48f5-8cf4-4fa41a3eb4c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:29.105022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d93031-02e4-48f5-8cf4-4fa41a3eb4c9 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.105392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d93031-02e4-48f5-8cf4-4fa41a3eb4c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.105677 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d93031-02e4-48f5-8cf4-4fa41a3eb4c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.113959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1029/2033] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:51:29.116221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d93031-02e4-48f5-8cf4-4fa41a3eb4c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:29.118095 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d93031-02e4-48f5-8cf4-4fa41a3eb4c9 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:29.120376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d846d5-e161-4b1d-aa86-f06d2f8ba235 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:29.131754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d846d5-e161-4b1d-aa86-f06d2f8ba235 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:29.135041 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1005/2034] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:51:29.137325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30c9a2-0877-4a5d-b0da-4e40bb153f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:29.137953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30c9a2-0877-4a5d-b0da-4e40bb153f53 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.138231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30c9a2-0877-4a5d-b0da-4e40bb153f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.138506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30c9a2-0877-4a5d-b0da-4e40bb153f53 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.139272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30c9a2-0877-4a5d-b0da-4e40bb153f53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:29.139603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30c9a2-0877-4a5d-b0da-4e40bb153f53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:29.141218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30c9a2-0877-4a5d-b0da-4e40bb153f53 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:29.157003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1030/2035] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:29.160296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d93031-02e4-48f5-8cf4-4fa41a3eb4c9 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:29.173943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d93031-02e4-48f5-8cf4-4fa41a3eb4c9 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:29.179804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6b7a8-5499-41f7-b148-8afeea8ad3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:29.182117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6b7a8-5499-41f7-b148-8afeea8ad3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.182345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6b7a8-5499-41f7-b148-8afeea8ad3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.182858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6b7a8-5499-41f7-b148-8afeea8ad3a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.184052 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6b7a8-5499-41f7-b148-8afeea8ad3a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:29.191051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6b7a8-5499-41f7-b148-8afeea8ad3a4 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:29.231777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1006/2036] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:51:29 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:29.237430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30c9a2-0877-4a5d-b0da-4e40bb153f53 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:29.253672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30c9a2-0877-4a5d-b0da-4e40bb153f53 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:29.341606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6c2b0-822a-45e1-8c86-5a3d4cdee846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:29.342239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c2b0-822a-45e1-8c86-5a3d4cdee846 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.342572 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c2b0-822a-45e1-8c86-5a3d4cdee846 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.343399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c2b0-822a-45e1-8c86-5a3d4cdee846 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.344225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6c2b0-822a-45e1-8c86-5a3d4cdee846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:29.346443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6c2b0-822a-45e1-8c86-5a3d4cdee846 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:29.392710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1031/2037] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:51:29 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:51:29.396529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6b7a8-5499-41f7-b148-8afeea8ad3a4 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:29.409377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6b7a8-5499-41f7-b148-8afeea8ad3a4 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:29.415469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355a359-9923-45d5-a1d9-952eb0b1bab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:29.416118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355a359-9923-45d5-a1d9-952eb0b1bab3 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.416404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355a359-9923-45d5-a1d9-952eb0b1bab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.416657 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355a359-9923-45d5-a1d9-952eb0b1bab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.417492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355a359-9923-45d5-a1d9-952eb0b1bab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:29.419111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355a359-9923-45d5-a1d9-952eb0b1bab3 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:29.431877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1007/2038] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:29.436371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6c2b0-822a-45e1-8c86-5a3d4cdee846 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:29.448601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6c2b0-822a-45e1-8c86-5a3d4cdee846 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:29.453743 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00221b0b-bc31-44e4-8e03-263dea379396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:29.454379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00221b0b-bc31-44e4-8e03-263dea379396 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.454685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00221b0b-bc31-44e4-8e03-263dea379396 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.454959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00221b0b-bc31-44e4-8e03-263dea379396 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.455669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00221b0b-bc31-44e4-8e03-263dea379396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:29.457291 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00221b0b-bc31-44e4-8e03-263dea379396 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:29.474175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1032/2039] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:51:29.478545 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355a359-9923-45d5-a1d9-952eb0b1bab3 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:29.490314 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355a359-9923-45d5-a1d9-952eb0b1bab3 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:29.499225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7b016-cd96-4da4-89aa-a4e9f3ae0a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:29.499914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7b016-cd96-4da4-89aa-a4e9f3ae0a47 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.500193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7b016-cd96-4da4-89aa-a4e9f3ae0a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.500468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7b016-cd96-4da4-89aa-a4e9f3ae0a47 admin admin] PATH_INFO: `/v3/projects/5c161b21fd204792884cb02e3d895c99/users/6b8e17d8c3d94e6fb28191c5988f8c6a/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.519157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7b016-cd96-4da4-89aa-a4e9f3ae0a47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c161b21fd204792884cb02e3d895c99, user_id=6b8e17d8c3d94e6fb28191c5988f8c6a, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:29.521896 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7b016-cd96-4da4-89aa-a4e9f3ae0a47 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:29.537650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1008/2040] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:29 2022] PUT /identity/v3/projects/5c161b21fd204792884cb02e3d895c99/users/6b8e17d8c3d94e6fb28191c5988f8c6a/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:29.549893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3a0a5-5277-404b-b84a-a5c69330d93d None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.550176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3a0a5-5277-404b-b84a-a5c69330d93d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.550449 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3a0a5-5277-404b-b84a-a5c69330d93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.581302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-2db3a0a5-5277-404b-b84a-a5c69330d93d None None] MFA Rules not processed for user `6b8e17d8c3d94e6fb28191c5988f8c6a`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:29.582586 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2db3a0a5-5277-404b-b84a-a5c69330d93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:29.666378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1033/2041] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:51:29.678361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7b016-cd96-4da4-89aa-a4e9f3ae0a47 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:29.681732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7fc7b016-cd96-4da4-89aa-a4e9f3ae0a47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:29.730583 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3a0a5-5277-404b-b84a-a5c69330d93d None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:29.731903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2db3a0a5-5277-404b-b84a-a5c69330d93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:29.754853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7b016-cd96-4da4-89aa-a4e9f3ae0a47 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:29.759260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f7c67-ff5f-4895-bf9a-aff5aed0de5e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72d4459164844e718544d7fde333db76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:29.759895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f7c67-ff5f-4895-bf9a-aff5aed0de5e service glance] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.760174 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f7c67-ff5f-4895-bf9a-aff5aed0de5e service glance] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.760607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f7c67-ff5f-4895-bf9a-aff5aed0de5e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.767808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:29.770235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f7c67-ff5f-4895-bf9a-aff5aed0de5e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:29.771722 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f7c67-ff5f-4895-bf9a-aff5aed0de5e service glance] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:29.782001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1009/2042] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:51:29.816559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3a0a5-5277-404b-b84a-a5c69330d93d None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:29.821999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e65b5a-60eb-4956-ba99-dd73463683c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:29.822829 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e65b5a-60eb-4956-ba99-dd73463683c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.823155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e65b5a-60eb-4956-ba99-dd73463683c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.823476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e65b5a-60eb-4956-ba99-dd73463683c1 admin admin] PATH_INFO: `/v3/projects/61e9cc35b8d242cf9165158977a3ccbd` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.824742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e65b5a-60eb-4956-ba99-dd73463683c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61e9cc35b8d242cf9165158977a3ccbd)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:29.826928 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e65b5a-60eb-4956-ba99-dd73463683c1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:29.918101 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-50e65b5a-60eb-4956-ba99-dd73463683c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61e9cc35b8d242cf9165158977a3ccbd', 'request_id': 'req-50e65b5a-60eb-4956-ba99-dd73463683c1'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:29.928000 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-50e65b5a-60eb-4956-ba99-dd73463683c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61e9cc35b8d242cf9165158977a3ccbd', 'request_id': 'req-50e65b5a-60eb-4956-ba99-dd73463683c1'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:29.942421 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-50e65b5a-60eb-4956-ba99-dd73463683c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61e9cc35b8d242cf9165158977a3ccbd', 'request_id': 'req-50e65b5a-60eb-4956-ba99-dd73463683c1'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:29.960995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-50e65b5a-60eb-4956-ba99-dd73463683c1 admin admin] The token cache is being invalidate because project 61e9cc35b8d242cf9165158977a3ccbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:51:29.962027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-50e65b5a-60eb-4956-ba99-dd73463683c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:29.963964 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-50e65b5a-60eb-4956-ba99-dd73463683c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:29.966154 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1034/2043] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:51:29 2022] DELETE /identity/v3/projects/61e9cc35b8d242cf9165158977a3ccbd => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:51:29.970994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a2aa-0053-4e80-80a6-096a1c8f62cc None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:29.971556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a2aa-0053-4e80-80a6-096a1c8f62cc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:29.971899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a2aa-0053-4e80-80a6-096a1c8f62cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:29.999146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-62a8a2aa-0053-4e80-80a6-096a1c8f62cc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:30.000706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-62a8a2aa-0053-4e80-80a6-096a1c8f62cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:30.036349 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:30.109402 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1010/2044] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:51:30.117372 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93318c69-29e2-4636-a564-0064033f9b1f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:30.117661 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93318c69-29e2-4636-a564-0064033f9b1f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:30.117998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93318c69-29e2-4636-a564-0064033f9b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:30.141530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-93318c69-29e2-4636-a564-0064033f9b1f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:30.142849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-93318c69-29e2-4636-a564-0064033f9b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:30.152902 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1035/2045] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:30.157004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8a2aa-0053-4e80-80a6-096a1c8f62cc None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:30.171463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8a2aa-0053-4e80-80a6-096a1c8f62cc None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:30.176499 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d717f39-3141-44bd-bbe0-304258fb8a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:30.177150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d717f39-3141-44bd-bbe0-304258fb8a76 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:30.177414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d717f39-3141-44bd-bbe0-304258fb8a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:30.177701 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d717f39-3141-44bd-bbe0-304258fb8a76 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:30.178377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d717f39-3141-44bd-bbe0-304258fb8a76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:30.178714 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d717f39-3141-44bd-bbe0-304258fb8a76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:30.180408 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d717f39-3141-44bd-bbe0-304258fb8a76 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:30.198700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1011/2046] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:30.202701 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93318c69-29e2-4636-a564-0064033f9b1f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:30.217669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93318c69-29e2-4636-a564-0064033f9b1f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:30.224544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2e744-54de-4f91-ba0d-93188d6abc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:30.224544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2e744-54de-4f91-ba0d-93188d6abc29 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:30.224826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2e744-54de-4f91-ba0d-93188d6abc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:30.225465 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2e744-54de-4f91-ba0d-93188d6abc29 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:30.225752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2e744-54de-4f91-ba0d-93188d6abc29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:30.228475 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2e744-54de-4f91-ba0d-93188d6abc29 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:30.264410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1036/2047] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:51:30 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:30.269067 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d717f39-3141-44bd-bbe0-304258fb8a76 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:30.280291 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d717f39-3141-44bd-bbe0-304258fb8a76 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:30.361510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f8054-0c79-4dad-8334-66e787ead074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:30.362141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f8054-0c79-4dad-8334-66e787ead074 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:30.362412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f8054-0c79-4dad-8334-66e787ead074 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:30.362713 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f8054-0c79-4dad-8334-66e787ead074 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:30.363388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f8054-0c79-4dad-8334-66e787ead074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:30.365479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f8054-0c79-4dad-8334-66e787ead074 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:30.394896 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1012/2048] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:51:30 2022] POST /identity/v3/users => generated 284 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:51:30.400485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2e744-54de-4f91-ba0d-93188d6abc29 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:30.411741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2e744-54de-4f91-ba0d-93188d6abc29 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:30.417657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ef61a-f63d-4772-82b5-35fd544aeb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:30.418284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ef61a-f63d-4772-82b5-35fd544aeb9b admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:30.418561 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ef61a-f63d-4772-82b5-35fd544aeb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:30.418857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ef61a-f63d-4772-82b5-35fd544aeb9b admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:30.419556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ef61a-f63d-4772-82b5-35fd544aeb9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:30.421252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ef61a-f63d-4772-82b5-35fd544aeb9b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:30.432523 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1037/2049] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:51:30.436771 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f8054-0c79-4dad-8334-66e787ead074 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:30.448018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f8054-0c79-4dad-8334-66e787ead074 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:30.453078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588d961-35ae-4581-aa89-86cd06193022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:30.453699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588d961-35ae-4581-aa89-86cd06193022 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:30.453976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588d961-35ae-4581-aa89-86cd06193022 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:30.454245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588d961-35ae-4581-aa89-86cd06193022 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:30.454974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588d961-35ae-4581-aa89-86cd06193022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:30.456600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588d961-35ae-4581-aa89-86cd06193022 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:30.469831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1013/2050] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:51:30.473851 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ef61a-f63d-4772-82b5-35fd544aeb9b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:30.484471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ef61a-f63d-4772-82b5-35fd544aeb9b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:30.489172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e953b3-4292-455e-89d3-4c99f737200e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:30.489831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e953b3-4292-455e-89d3-4c99f737200e admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:30.490113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e953b3-4292-455e-89d3-4c99f737200e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:30.490382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e953b3-4292-455e-89d3-4c99f737200e admin admin] PATH_INFO: `/v3/projects/7e5d947dbe804f0db311a37d9cca6c5a/users/1a816b1728944b44951306ff3b23abfa/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:30.506229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e953b3-4292-455e-89d3-4c99f737200e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5d947dbe804f0db311a37d9cca6c5a, user_id=1a816b1728944b44951306ff3b23abfa, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:30.509613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e953b3-4292-455e-89d3-4c99f737200e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:30.525104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1038/2051] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:30 2022] PUT /identity/v3/projects/7e5d947dbe804f0db311a37d9cca6c5a/users/1a816b1728944b44951306ff3b23abfa/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:30.535455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9210bd-2d75-4cc4-9b1a-514b98c6e3e8 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:30.535741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9210bd-2d75-4cc4-9b1a-514b98c6e3e8 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:30.536014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9210bd-2d75-4cc4-9b1a-514b98c6e3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:30.565047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-9b9210bd-2d75-4cc4-9b1a-514b98c6e3e8 None None] MFA Rules not processed for user `1a816b1728944b44951306ff3b23abfa`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:30.566273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9b9210bd-2d75-4cc4-9b1a-514b98c6e3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:30.632824 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1014/2052] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:30.647393 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e953b3-4292-455e-89d3-4c99f737200e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:30.650684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b3e953b3-4292-455e-89d3-4c99f737200e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:30.705304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:30.708864 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e953b3-4292-455e-89d3-4c99f737200e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:30.713625 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a5c31-686b-4bb4-8ddd-d0620ed769a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72d4459164844e718544d7fde333db76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:30.714242 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a5c31-686b-4bb4-8ddd-d0620ed769a6 service glance] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:30.714519 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a5c31-686b-4bb4-8ddd-d0620ed769a6 service glance] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:30.714826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a5c31-686b-4bb4-8ddd-d0620ed769a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:30.723857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a5c31-686b-4bb4-8ddd-d0620ed769a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:30.725287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a5c31-686b-4bb4-8ddd-d0620ed769a6 service glance] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:30.735152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1039/2053] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:51:40.629785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:40.730596 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:51.802348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9210bd-2d75-4cc4-9b1a-514b98c6e3e8 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:51.803807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9b9210bd-2d75-4cc4-9b1a-514b98c6e3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:51.874735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9210bd-2d75-4cc4-9b1a-514b98c6e3e8 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:51.879272 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:51.879925 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:51.880208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:51.880477 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] PATH_INFO: `/v3/users/8b7e9108ed964ab1b4b3e900831ea31b` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:51.881685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b7e9108ed964ab1b4b3e900831ea31b)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:51.883659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:51.934253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b7e9108ed964ab1b4b3e900831ea31b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:51.942061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b7e9108ed964ab1b4b3e900831ea31b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:51.943337 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b7e9108ed964ab1b4b3e900831ea31b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:51.957023 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b7e9108ed964ab1b4b3e900831ea31b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:51.966449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b7e9108ed964ab1b4b3e900831ea31b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:51.974386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1015/2054] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:51:51 2022] DELETE /identity/v3/users/8b7e9108ed964ab1b4b3e900831ea31b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:51.980438 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7a4df-e633-42dd-8c32-fc8da15cb352 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:51.980944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7a4df-e633-42dd-8c32-fc8da15cb352 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:51.981062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7a4df-e633-42dd-8c32-fc8da15cb352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:52.003478 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-a1b7a4df-e633-42dd-8c32-fc8da15cb352 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:52.004707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a1b7a4df-e633-42dd-8c32-fc8da15cb352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:52.076622 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1040/2055] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:52.086738 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:52.089344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:52.158487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c39d3-94c7-4354-84da-c3ac5595c6ae admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:52.162331 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657af58-6e01-4dff-84ce-6e67f261a8f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:52.163303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657af58-6e01-4dff-84ce-6e67f261a8f5 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:52.163692 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657af58-6e01-4dff-84ce-6e67f261a8f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:52.163969 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657af58-6e01-4dff-84ce-6e67f261a8f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:52.173940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657af58-6e01-4dff-84ce-6e67f261a8f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:52.175439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657af58-6e01-4dff-84ce-6e67f261a8f5 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:52.187564 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1016/2056] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:51:52.542397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7a4df-e633-42dd-8c32-fc8da15cb352 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:52.544066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a1b7a4df-e633-42dd-8c32-fc8da15cb352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:52.558044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7a4df-e633-42dd-8c32-fc8da15cb352 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:52.567059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:52.570522 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:52.570827 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:52.571105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] PATH_INFO: `/v3/projects/ba711a7c6df74c6da770803cc2ef8d6f` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:52.573159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba711a7c6df74c6da770803cc2ef8d6f)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:52.575138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:52.642335 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba711a7c6df74c6da770803cc2ef8d6f', 'request_id': 'req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:52.649185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba711a7c6df74c6da770803cc2ef8d6f', 'request_id': 'req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:52.660133 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba711a7c6df74c6da770803cc2ef8d6f', 'request_id': 'req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:52.671929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] The token cache is being invalidate because project ba711a7c6df74c6da770803cc2ef8d6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:51:52.672215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:52.672874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:52.674422 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1041/2057] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:51:52 2022] DELETE /identity/v3/projects/ba711a7c6df74c6da770803cc2ef8d6f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:52.679987 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657af58-6e01-4dff-84ce-6e67f261a8f5 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:52.691717 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657af58-6e01-4dff-84ce-6e67f261a8f5 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:52.714719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:52.764992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674bdb2-bdb0-4218-8f90-e7438eb2c249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:52.765649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674bdb2-bdb0-4218-8f90-e7438eb2c249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:52.765928 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674bdb2-bdb0-4218-8f90-e7438eb2c249 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:52.766201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674bdb2-bdb0-4218-8f90-e7438eb2c249 admin admin] PATH_INFO: `/v3/users/ff5a4265507644dfa4d9ccaed1ab5d83` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:52.768117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674bdb2-bdb0-4218-8f90-e7438eb2c249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff5a4265507644dfa4d9ccaed1ab5d83)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:52.769500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674bdb2-bdb0-4218-8f90-e7438eb2c249 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:52.832997 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-3674bdb2-bdb0-4218-8f90-e7438eb2c249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff5a4265507644dfa4d9ccaed1ab5d83'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:52.841829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-3674bdb2-bdb0-4218-8f90-e7438eb2c249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff5a4265507644dfa4d9ccaed1ab5d83'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:52.842573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-3674bdb2-bdb0-4218-8f90-e7438eb2c249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff5a4265507644dfa4d9ccaed1ab5d83'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:52.856100 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-3674bdb2-bdb0-4218-8f90-e7438eb2c249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff5a4265507644dfa4d9ccaed1ab5d83'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:52.864905 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-3674bdb2-bdb0-4218-8f90-e7438eb2c249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff5a4265507644dfa4d9ccaed1ab5d83'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:52.873895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1017/2058] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:51:52 2022] DELETE /identity/v3/users/ff5a4265507644dfa4d9ccaed1ab5d83 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:53.174960 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:53.176482 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:53.192776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be762d2-1d63-46db-94ea-ecd7ce36fa9f admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:53.261517 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400aef4-9385-44fd-98af-516c5aa5145d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:53.262228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400aef4-9385-44fd-98af-516c5aa5145d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:53.262507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400aef4-9385-44fd-98af-516c5aa5145d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:53.262825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400aef4-9385-44fd-98af-516c5aa5145d admin admin] PATH_INFO: `/v3/projects/a6eae51ada44491e851e1c80f953e0eb` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:53.264066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400aef4-9385-44fd-98af-516c5aa5145d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6eae51ada44491e851e1c80f953e0eb)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:53.266035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400aef4-9385-44fd-98af-516c5aa5145d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:53.305223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:53.328022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-7400aef4-9385-44fd-98af-516c5aa5145d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6eae51ada44491e851e1c80f953e0eb', 'request_id': 'req-7400aef4-9385-44fd-98af-516c5aa5145d'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:53.334322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-7400aef4-9385-44fd-98af-516c5aa5145d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6eae51ada44491e851e1c80f953e0eb', 'request_id': 'req-7400aef4-9385-44fd-98af-516c5aa5145d'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:53.343644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-7400aef4-9385-44fd-98af-516c5aa5145d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6eae51ada44491e851e1c80f953e0eb', 'request_id': 'req-7400aef4-9385-44fd-98af-516c5aa5145d'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:53.358736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-7400aef4-9385-44fd-98af-516c5aa5145d admin admin] The token cache is being invalidate because project a6eae51ada44491e851e1c80f953e0eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:51:53.359028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-7400aef4-9385-44fd-98af-516c5aa5145d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:53.359694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-7400aef4-9385-44fd-98af-516c5aa5145d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:53.361233 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1042/2059] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:51:53 2022] DELETE /identity/v3/projects/a6eae51ada44491e851e1c80f953e0eb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:53.366141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f335855-cf16-4505-b604-4d7240624683 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:53.366433 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f335855-cf16-4505-b604-4d7240624683 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:53.366749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f335855-cf16-4505-b604-4d7240624683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:53.388470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-0f335855-cf16-4505-b604-4d7240624683 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:53.389699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-0f335855-cf16-4505-b604-4d7240624683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:53.455679 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1018/2060] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:53.465591 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9806f7-574f-4a76-b221-57003992ec1d None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:53.465958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9806f7-574f-4a76-b221-57003992ec1d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:53.466785 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9806f7-574f-4a76-b221-57003992ec1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:53.488113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-eb9806f7-574f-4a76-b221-57003992ec1d None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:53.489329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-eb9806f7-574f-4a76-b221-57003992ec1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:53.497903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1043/2061] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:51:53.502617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f335855-cf16-4505-b604-4d7240624683 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:53.513235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f335855-cf16-4505-b604-4d7240624683 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:53.517988 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05aa21-353f-4d99-bb66-f13c74b500c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:53.518606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05aa21-353f-4d99-bb66-f13c74b500c6 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:53.518915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05aa21-353f-4d99-bb66-f13c74b500c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:53.519187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05aa21-353f-4d99-bb66-f13c74b500c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:53.519883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05aa21-353f-4d99-bb66-f13c74b500c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:53.520186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05aa21-353f-4d99-bb66-f13c74b500c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:53.521800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05aa21-353f-4d99-bb66-f13c74b500c6 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:53.537924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1019/2062] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:53.542386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9806f7-574f-4a76-b221-57003992ec1d None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:53.552390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9806f7-574f-4a76-b221-57003992ec1d None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:53.557811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb131a-762e-4913-b363-9ae88ca0b44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:53.558434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb131a-762e-4913-b363-9ae88ca0b44e admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:53.558759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb131a-762e-4913-b363-9ae88ca0b44e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:53.559014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb131a-762e-4913-b363-9ae88ca0b44e admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:53.559678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb131a-762e-4913-b363-9ae88ca0b44e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:53.561585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb131a-762e-4913-b363-9ae88ca0b44e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:53.591979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1044/2063] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:51:53 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:53.594995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05aa21-353f-4d99-bb66-f13c74b500c6 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:53.605733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05aa21-353f-4d99-bb66-f13c74b500c6 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:53.644523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:53.683015 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0645a2-05b1-46b2-b2eb-9c9f7b805be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:53.683644 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0645a2-05b1-46b2-b2eb-9c9f7b805be9 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:53.683921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0645a2-05b1-46b2-b2eb-9c9f7b805be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:53.684193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0645a2-05b1-46b2-b2eb-9c9f7b805be9 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:53.684868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0645a2-05b1-46b2-b2eb-9c9f7b805be9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:53.686809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0645a2-05b1-46b2-b2eb-9c9f7b805be9 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:53.713005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1020/2064] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:51:53 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:53.714199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb131a-762e-4913-b363-9ae88ca0b44e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:53.725476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb131a-762e-4913-b363-9ae88ca0b44e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:53.731509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463f956-e750-4990-9b62-3f6551eaaea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:53.732155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463f956-e750-4990-9b62-3f6551eaaea0 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:53.732434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463f956-e750-4990-9b62-3f6551eaaea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:53.732706 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463f956-e750-4990-9b62-3f6551eaaea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:53.733413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463f956-e750-4990-9b62-3f6551eaaea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:53.735061 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463f956-e750-4990-9b62-3f6551eaaea0 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:53.746491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1045/2065] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:53.752611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0645a2-05b1-46b2-b2eb-9c9f7b805be9 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:53.764049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0645a2-05b1-46b2-b2eb-9c9f7b805be9 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:53.769277 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5cb76-40b7-4689-acb3-d186870d5205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:53.769934 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5cb76-40b7-4689-acb3-d186870d5205 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:53.770212 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5cb76-40b7-4689-acb3-d186870d5205 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:53.770484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5cb76-40b7-4689-acb3-d186870d5205 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:53.771233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5cb76-40b7-4689-acb3-d186870d5205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:53.772836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5cb76-40b7-4689-acb3-d186870d5205 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:53.784305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1021/2066] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:53.788754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463f956-e750-4990-9b62-3f6551eaaea0 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:53.802254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463f956-e750-4990-9b62-3f6551eaaea0 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:53.808937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e3a6e-67d2-4502-ab41-d37fa919a289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:53.809814 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e3a6e-67d2-4502-ab41-d37fa919a289 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:53.810086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e3a6e-67d2-4502-ab41-d37fa919a289 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:53.810358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e3a6e-67d2-4502-ab41-d37fa919a289 admin admin] PATH_INFO: `/v3/projects/80f74f12faa44520b55fd0726b5e4acc/users/6db40250582a4c918b4a1fac131a3826/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:53.827655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e3a6e-67d2-4502-ab41-d37fa919a289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80f74f12faa44520b55fd0726b5e4acc, user_id=6db40250582a4c918b4a1fac131a3826, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:53.830287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e3a6e-67d2-4502-ab41-d37fa919a289 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:53.847258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1046/2067] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:53 2022] PUT /identity/v3/projects/80f74f12faa44520b55fd0726b5e4acc/users/6db40250582a4c918b4a1fac131a3826/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:53.861626 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59792e4-22d2-4b96-91d3-63e18a93459c None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:53.862301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59792e4-22d2-4b96-91d3-63e18a93459c None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:53.862301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59792e4-22d2-4b96-91d3-63e18a93459c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:53.897475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-a59792e4-22d2-4b96-91d3-63e18a93459c None None] MFA Rules not processed for user `6db40250582a4c918b4a1fac131a3826`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:53.899096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a59792e4-22d2-4b96-91d3-63e18a93459c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:53.988257 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1022/2068] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:53.992874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e3a6e-67d2-4502-ab41-d37fa919a289 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:54.069050 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e3a6e-67d2-4502-ab41-d37fa919a289 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:54.073729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62188650-64d8-494d-ba09-aad14ddc4ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:54.074354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62188650-64d8-494d-ba09-aad14ddc4ecc admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:54.074652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62188650-64d8-494d-ba09-aad14ddc4ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:54.074931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62188650-64d8-494d-ba09-aad14ddc4ecc admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:54.075612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62188650-64d8-494d-ba09-aad14ddc4ecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:54.077536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62188650-64d8-494d-ba09-aad14ddc4ecc admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:54.107399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1047/2069] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:51:53 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:54.111458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59792e4-22d2-4b96-91d3-63e18a93459c None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:54.122879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59792e4-22d2-4b96-91d3-63e18a93459c None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:54.180173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ea444-5730-41fb-a7d8-1003309d1e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:54.181070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ea444-5730-41fb-a7d8-1003309d1e21 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:54.181346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ea444-5730-41fb-a7d8-1003309d1e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:54.181622 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ea444-5730-41fb-a7d8-1003309d1e21 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:54.182371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ea444-5730-41fb-a7d8-1003309d1e21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:54.184337 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ea444-5730-41fb-a7d8-1003309d1e21 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:54.209085 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1023/2070] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:51:54 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:54.214115 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62188650-64d8-494d-ba09-aad14ddc4ecc admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:54.224969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62188650-64d8-494d-ba09-aad14ddc4ecc admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:54.231157 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f1b6e-3ffa-497e-85a5-6de304a8062a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:54.232033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f1b6e-3ffa-497e-85a5-6de304a8062a admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:54.232400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f1b6e-3ffa-497e-85a5-6de304a8062a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:54.232775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f1b6e-3ffa-497e-85a5-6de304a8062a admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:54.233829 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f1b6e-3ffa-497e-85a5-6de304a8062a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:54.235595 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f1b6e-3ffa-497e-85a5-6de304a8062a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:54.247086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1048/2071] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:51:54.251625 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ea444-5730-41fb-a7d8-1003309d1e21 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:54.263346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ea444-5730-41fb-a7d8-1003309d1e21 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:54.268413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c4ce9-1d88-466d-a8a7-be5aad95b180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:54.269090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c4ce9-1d88-466d-a8a7-be5aad95b180 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:54.269368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c4ce9-1d88-466d-a8a7-be5aad95b180 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:54.269639 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c4ce9-1d88-466d-a8a7-be5aad95b180 admin admin] PATH_INFO: `/v3/projects/e00bfe94ea50411784503718ee02defc/users/2e930d763fa74c478fd8dee11c101907/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:54.286223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c4ce9-1d88-466d-a8a7-be5aad95b180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e00bfe94ea50411784503718ee02defc, user_id=2e930d763fa74c478fd8dee11c101907, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:54.288922 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c4ce9-1d88-466d-a8a7-be5aad95b180 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:54.305095 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1024/2072] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:54 2022] PUT /identity/v3/projects/e00bfe94ea50411784503718ee02defc/users/2e930d763fa74c478fd8dee11c101907/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:51:54.310385 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f1b6e-3ffa-497e-85a5-6de304a8062a admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:54.320815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f1b6e-3ffa-497e-85a5-6de304a8062a admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:54.349563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:54.396832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec65c3c-1968-4502-b57b-005a8c5827aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:54.397440 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec65c3c-1968-4502-b57b-005a8c5827aa admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:54.397716 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec65c3c-1968-4502-b57b-005a8c5827aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:54.397986 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec65c3c-1968-4502-b57b-005a8c5827aa admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:54.398731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec65c3c-1968-4502-b57b-005a8c5827aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:54.401342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec65c3c-1968-4502-b57b-005a8c5827aa admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:54.417671 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1049/2073] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:54.422075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c4ce9-1d88-466d-a8a7-be5aad95b180 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:54.427786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:54.435882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c4ce9-1d88-466d-a8a7-be5aad95b180 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:54.441056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fdad4-3008-418d-ad20-da3d45a015ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:54.441727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fdad4-3008-418d-ad20-da3d45a015ed admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:54.442005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fdad4-3008-418d-ad20-da3d45a015ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:54.442276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fdad4-3008-418d-ad20-da3d45a015ed admin admin] PATH_INFO: `/v3/projects/e00bfe94ea50411784503718ee02defc/users/2e930d763fa74c478fd8dee11c101907/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:54.444511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fdad4-3008-418d-ad20-da3d45a015ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=e00bfe94ea50411784503718ee02defc, user_id=2e930d763fa74c478fd8dee11c101907, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:54.447774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fdad4-3008-418d-ad20-da3d45a015ed admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:54.458798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1025/2074] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:54 2022] PUT /identity/v3/projects/e00bfe94ea50411784503718ee02defc/users/2e930d763fa74c478fd8dee11c101907/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:54.470192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a87ff8-bee1-4847-a4f7-4078d5e373ac None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:54.470522 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a87ff8-bee1-4847-a4f7-4078d5e373ac None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:54.470776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a87ff8-bee1-4847-a4f7-4078d5e373ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:54.502399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-74a87ff8-bee1-4847-a4f7-4078d5e373ac None None] MFA Rules not processed for user `2e930d763fa74c478fd8dee11c101907`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:54.503672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-74a87ff8-bee1-4847-a4f7-4078d5e373ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:54.590414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1050/2075] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:54.601492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fdad4-3008-418d-ad20-da3d45a015ed admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:54.603276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-242fdad4-3008-418d-ad20-da3d45a015ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:54.664704 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fdad4-3008-418d-ad20-da3d45a015ed admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:54.668751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce2adb-1cc8-42b0-bade-44e5cf9db116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:54.669372 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce2adb-1cc8-42b0-bade-44e5cf9db116 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:54.669650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce2adb-1cc8-42b0-bade-44e5cf9db116 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:54.669920 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce2adb-1cc8-42b0-bade-44e5cf9db116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:54.679685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce2adb-1cc8-42b0-bade-44e5cf9db116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:54.681949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce2adb-1cc8-42b0-bade-44e5cf9db116 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:54.692742 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1026/2076] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:51:54.797736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a87ff8-bee1-4847-a4f7-4078d5e373ac None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:54.799156 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-74a87ff8-bee1-4847-a4f7-4078d5e373ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:54.813593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a87ff8-bee1-4847-a4f7-4078d5e373ac None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:54.817217 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9832b-9582-4fe0-b996-57cb904158b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:54.817833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9832b-9582-4fe0-b996-57cb904158b1 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:54.818109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9832b-9582-4fe0-b996-57cb904158b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:54.818375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9832b-9582-4fe0-b996-57cb904158b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:54.879554 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9832b-9582-4fe0-b996-57cb904158b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:54.880879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9832b-9582-4fe0-b996-57cb904158b1 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:54.891296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1051/2077] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:51:57.533143 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce2adb-1cc8-42b0-bade-44e5cf9db116 service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:57.604409 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce2adb-1cc8-42b0-bade-44e5cf9db116 service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:57.608879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:57.609524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:57.609806 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:57.610076 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] PATH_INFO: `/v3/users/6db40250582a4c918b4a1fac131a3826` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:57.611276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6db40250582a4c918b4a1fac131a3826)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:57.613216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:57.674287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6db40250582a4c918b4a1fac131a3826'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:57.685283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6db40250582a4c918b4a1fac131a3826'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:57.686030 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6db40250582a4c918b4a1fac131a3826'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:57.701814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6db40250582a4c918b4a1fac131a3826'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:57.728425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6db40250582a4c918b4a1fac131a3826'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:57.735069 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1027/2078] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:51:57 2022] DELETE /identity/v3/users/6db40250582a4c918b4a1fac131a3826 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:57.739687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c833a-cd12-43d3-805a-9aed1c51579a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:57.740163 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c833a-cd12-43d3-805a-9aed1c51579a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:57.740259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c833a-cd12-43d3-805a-9aed1c51579a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:57.761206 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c35c833a-cd12-43d3-805a-9aed1c51579a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:57.762431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c35c833a-cd12-43d3-805a-9aed1c51579a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:57.809741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:57.830030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1052/2079] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:57.839972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:57.841491 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:57.858778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:57.919072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a2035-0610-4ca8-8e25-9f6eba9d475b admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:57.922792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134ebcc-aee1-40d2-933c-79b6b5559427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:57.923412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134ebcc-aee1-40d2-933c-79b6b5559427 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:57.923688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134ebcc-aee1-40d2-933c-79b6b5559427 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:57.923959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134ebcc-aee1-40d2-933c-79b6b5559427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:57.934553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134ebcc-aee1-40d2-933c-79b6b5559427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:57.935639 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134ebcc-aee1-40d2-933c-79b6b5559427 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:57.945773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1028/2080] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:51:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:51:58.248079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c833a-cd12-43d3-805a-9aed1c51579a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:58.249542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c35c833a-cd12-43d3-805a-9aed1c51579a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:58.273950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c833a-cd12-43d3-805a-9aed1c51579a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:58.282831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:58.283393 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:58.283684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:58.283934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] PATH_INFO: `/v3/projects/80f74f12faa44520b55fd0726b5e4acc` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:58.285122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80f74f12faa44520b55fd0726b5e4acc)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:58.287070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:58.344565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80f74f12faa44520b55fd0726b5e4acc', 'request_id': 'req-b198cc65-e666-4af0-b20b-621073b53b17'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:58.351414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80f74f12faa44520b55fd0726b5e4acc', 'request_id': 'req-b198cc65-e666-4af0-b20b-621073b53b17'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:58.360068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80f74f12faa44520b55fd0726b5e4acc', 'request_id': 'req-b198cc65-e666-4af0-b20b-621073b53b17'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:58.374050 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] The token cache is being invalidate because project 80f74f12faa44520b55fd0726b5e4acc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:51:58.374324 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:58.375088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:58.376847 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1053/2081] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:51:58 2022] DELETE /identity/v3/projects/80f74f12faa44520b55fd0726b5e4acc => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:58.381030 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134ebcc-aee1-40d2-933c-79b6b5559427 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:58.392275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134ebcc-aee1-40d2-933c-79b6b5559427 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:58.463631 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847d0ef-88f7-4b82-bd69-1d5c8b00fc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:58.464092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847d0ef-88f7-4b82-bd69-1d5c8b00fc0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:58.464362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847d0ef-88f7-4b82-bd69-1d5c8b00fc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:58.464639 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847d0ef-88f7-4b82-bd69-1d5c8b00fc0b admin admin] PATH_INFO: `/v3/users/2e930d763fa74c478fd8dee11c101907` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:58.465840 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847d0ef-88f7-4b82-bd69-1d5c8b00fc0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e930d763fa74c478fd8dee11c101907)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:58.467826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847d0ef-88f7-4b82-bd69-1d5c8b00fc0b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:58.517896 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b847d0ef-88f7-4b82-bd69-1d5c8b00fc0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e930d763fa74c478fd8dee11c101907'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:58.526448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b847d0ef-88f7-4b82-bd69-1d5c8b00fc0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e930d763fa74c478fd8dee11c101907'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:58.526929 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b847d0ef-88f7-4b82-bd69-1d5c8b00fc0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e930d763fa74c478fd8dee11c101907'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:58.540013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b847d0ef-88f7-4b82-bd69-1d5c8b00fc0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e930d763fa74c478fd8dee11c101907'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:58.550141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b847d0ef-88f7-4b82-bd69-1d5c8b00fc0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e930d763fa74c478fd8dee11c101907'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:58.557319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1029/2082] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:51:58 2022] DELETE /identity/v3/users/2e930d763fa74c478fd8dee11c101907 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:58.872688 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:58.880465 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:58.888173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b198cc65-e666-4af0-b20b-621073b53b17 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:58.963814 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761604ae-f380-4c69-b9f8-975fae72d004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:58.964718 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761604ae-f380-4c69-b9f8-975fae72d004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:58.965131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761604ae-f380-4c69-b9f8-975fae72d004 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:58.965413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761604ae-f380-4c69-b9f8-975fae72d004 admin admin] PATH_INFO: `/v3/projects/e00bfe94ea50411784503718ee02defc` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:58.966692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761604ae-f380-4c69-b9f8-975fae72d004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e00bfe94ea50411784503718ee02defc)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:58.968792 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761604ae-f380-4c69-b9f8-975fae72d004 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:59.021482 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-761604ae-f380-4c69-b9f8-975fae72d004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e00bfe94ea50411784503718ee02defc', 'request_id': 'req-761604ae-f380-4c69-b9f8-975fae72d004'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:59.028450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-761604ae-f380-4c69-b9f8-975fae72d004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e00bfe94ea50411784503718ee02defc', 'request_id': 'req-761604ae-f380-4c69-b9f8-975fae72d004'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:59.036741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-761604ae-f380-4c69-b9f8-975fae72d004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e00bfe94ea50411784503718ee02defc', 'request_id': 'req-761604ae-f380-4c69-b9f8-975fae72d004'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:59.044716 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:59.060218 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-761604ae-f380-4c69-b9f8-975fae72d004 admin admin] The token cache is being invalidate because project e00bfe94ea50411784503718ee02defc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:51:59.060820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-761604ae-f380-4c69-b9f8-975fae72d004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:59.061957 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-761604ae-f380-4c69-b9f8-975fae72d004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:51:59.064640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1054/2083] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:51:58 2022] DELETE /identity/v3/projects/e00bfe94ea50411784503718ee02defc => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:59.069654 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05691949-91f0-445a-b010-7cb1039e0f25 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.069937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05691949-91f0-445a-b010-7cb1039e0f25 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.070208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05691949-91f0-445a-b010-7cb1039e0f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.091481 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-05691949-91f0-445a-b010-7cb1039e0f25 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:59.092706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-05691949-91f0-445a-b010-7cb1039e0f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:59.129607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:59.172233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1030/2084] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:51:59.181426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575bbf7-e439-436f-8b08-89ec3f5c1a3a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.181917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575bbf7-e439-436f-8b08-89ec3f5c1a3a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.182312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575bbf7-e439-436f-8b08-89ec3f5c1a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.204570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c575bbf7-e439-436f-8b08-89ec3f5c1a3a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:59.205996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c575bbf7-e439-436f-8b08-89ec3f5c1a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:59.218346 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1055/2085] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:51:59.222923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05691949-91f0-445a-b010-7cb1039e0f25 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:59.234935 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05691949-91f0-445a-b010-7cb1039e0f25 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:59.239773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7d63c-7dfc-40ab-afc5-8e230694ffe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:59.240389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7d63c-7dfc-40ab-afc5-8e230694ffe1 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.240665 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7d63c-7dfc-40ab-afc5-8e230694ffe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.240935 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7d63c-7dfc-40ab-afc5-8e230694ffe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.241686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7d63c-7dfc-40ab-afc5-8e230694ffe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:59.241993 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7d63c-7dfc-40ab-afc5-8e230694ffe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:59.243637 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7d63c-7dfc-40ab-afc5-8e230694ffe1 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:59.257208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1031/2086] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:59.261688 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575bbf7-e439-436f-8b08-89ec3f5c1a3a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:59.271695 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575bbf7-e439-436f-8b08-89ec3f5c1a3a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:59.277821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337e0c6-51fc-4f35-aac0-7c0717617474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:59.278482 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337e0c6-51fc-4f35-aac0-7c0717617474 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.278823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337e0c6-51fc-4f35-aac0-7c0717617474 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.279096 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337e0c6-51fc-4f35-aac0-7c0717617474 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.279799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337e0c6-51fc-4f35-aac0-7c0717617474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:59.281725 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337e0c6-51fc-4f35-aac0-7c0717617474 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:59.311752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1056/2087] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:51:59 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:59.316662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7d63c-7dfc-40ab-afc5-8e230694ffe1 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:59.330984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7d63c-7dfc-40ab-afc5-8e230694ffe1 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:59.390953 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fb0a3-7748-4f5a-a003-a6545dac3426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:59.391572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fb0a3-7748-4f5a-a003-a6545dac3426 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.391854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fb0a3-7748-4f5a-a003-a6545dac3426 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.392131 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fb0a3-7748-4f5a-a003-a6545dac3426 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.392811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fb0a3-7748-4f5a-a003-a6545dac3426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:59.394752 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fb0a3-7748-4f5a-a003-a6545dac3426 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:59.418725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1032/2088] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:51:59 2022] POST /identity/v3/users => generated 305 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:51:59.428820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337e0c6-51fc-4f35-aac0-7c0717617474 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:59.435222 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337e0c6-51fc-4f35-aac0-7c0717617474 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:59.440355 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b54e4-fec3-4df0-b66c-b7ccc446f0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:59.440981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b54e4-fec3-4df0-b66c-b7ccc446f0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.441264 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b54e4-fec3-4df0-b66c-b7ccc446f0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.441539 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b54e4-fec3-4df0-b66c-b7ccc446f0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.442251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b54e4-fec3-4df0-b66c-b7ccc446f0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:59.443905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b54e4-fec3-4df0-b66c-b7ccc446f0f4 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:59.456021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1057/2089] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:59.460176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fb0a3-7748-4f5a-a003-a6545dac3426 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:59.471526 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fb0a3-7748-4f5a-a003-a6545dac3426 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:59.476967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e37b48-32eb-4078-9f1e-841e1d2c537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:59.477666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e37b48-32eb-4078-9f1e-841e1d2c537e admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.478507 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e37b48-32eb-4078-9f1e-841e1d2c537e admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.478810 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e37b48-32eb-4078-9f1e-841e1d2c537e admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.479449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e37b48-32eb-4078-9f1e-841e1d2c537e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:59.481058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e37b48-32eb-4078-9f1e-841e1d2c537e admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:59.492904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1033/2090] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:51:59.497342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b54e4-fec3-4df0-b66c-b7ccc446f0f4 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:59.509959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b54e4-fec3-4df0-b66c-b7ccc446f0f4 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:59.514977 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feba626-cb7b-4729-9727-3384dce1b251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:59.515654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feba626-cb7b-4729-9727-3384dce1b251 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.515931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feba626-cb7b-4729-9727-3384dce1b251 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.516240 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feba626-cb7b-4729-9727-3384dce1b251 admin admin] PATH_INFO: `/v3/projects/06812155ebab467782d121c66d15496c/users/3e52eeffb8cf424d885783c511d88183/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.547496 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feba626-cb7b-4729-9727-3384dce1b251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06812155ebab467782d121c66d15496c, user_id=3e52eeffb8cf424d885783c511d88183, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:59.547496 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feba626-cb7b-4729-9727-3384dce1b251 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:59.563059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1058/2091] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:59 2022] PUT /identity/v3/projects/06812155ebab467782d121c66d15496c/users/3e52eeffb8cf424d885783c511d88183/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:59.576488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3b591-3544-458c-bc6b-c15af2126e90 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.577029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3b591-3544-458c-bc6b-c15af2126e90 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.577029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e3b591-3544-458c-bc6b-c15af2126e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.608263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-14e3b591-3544-458c-bc6b-c15af2126e90 None None] MFA Rules not processed for user `3e52eeffb8cf424d885783c511d88183`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:59.609487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-14e3b591-3544-458c-bc6b-c15af2126e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:59.694330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1034/2092] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:51:59 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:51:59.699762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feba626-cb7b-4729-9727-3384dce1b251 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:59.767871 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feba626-cb7b-4729-9727-3384dce1b251 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:59.772533 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cea5b-b25e-4f93-b0a2-c733132dfc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:59.773139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cea5b-b25e-4f93-b0a2-c733132dfc57 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.773411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cea5b-b25e-4f93-b0a2-c733132dfc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.773674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cea5b-b25e-4f93-b0a2-c733132dfc57 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.774649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cea5b-b25e-4f93-b0a2-c733132dfc57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:59.776690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cea5b-b25e-4f93-b0a2-c733132dfc57 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:59.812881 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1059/2093] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:51:59 2022] POST /identity/v3/projects => generated 409 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:59.814331 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3b591-3544-458c-bc6b-c15af2126e90 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:59.827796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e3b591-3544-458c-bc6b-c15af2126e90 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:59.858502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:59.895855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d2605-5526-4551-9d64-81fbfeb38f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:59.896482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d2605-5526-4551-9d64-81fbfeb38f9c admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.896812 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d2605-5526-4551-9d64-81fbfeb38f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.897122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d2605-5526-4551-9d64-81fbfeb38f9c admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.897952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d2605-5526-4551-9d64-81fbfeb38f9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:59.900483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d2605-5526-4551-9d64-81fbfeb38f9c admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:59.925319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1035/2094] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:51:59 2022] POST /identity/v3/users => generated 311 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:59.929452 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cea5b-b25e-4f93-b0a2-c733132dfc57 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:59.940223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cea5b-b25e-4f93-b0a2-c733132dfc57 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:59.947397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4aac8-d714-4a89-995d-009fdb5c5e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:59.948030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4aac8-d714-4a89-995d-009fdb5c5e51 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.948309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4aac8-d714-4a89-995d-009fdb5c5e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.948580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4aac8-d714-4a89-995d-009fdb5c5e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:59.949286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4aac8-d714-4a89-995d-009fdb5c5e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:59.950907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4aac8-d714-4a89-995d-009fdb5c5e51 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:59.963251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1060/2095] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:51:59.968342 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d2605-5526-4551-9d64-81fbfeb38f9c admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:59.980791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d2605-5526-4551-9d64-81fbfeb38f9c admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:59.985689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd976b-6595-4755-8ddc-38bdf6480b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:59.986362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd976b-6595-4755-8ddc-38bdf6480b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:59.986654 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd976b-6595-4755-8ddc-38bdf6480b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:59.986940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd976b-6595-4755-8ddc-38bdf6480b17 admin admin] PATH_INFO: `/v3/projects/6994396054cd464d8ac0668b3320a337/users/74c1232b400f4ccabe88a14199756686/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:00.008373 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd976b-6595-4755-8ddc-38bdf6480b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6994396054cd464d8ac0668b3320a337, user_id=74c1232b400f4ccabe88a14199756686, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:00.013787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd976b-6595-4755-8ddc-38bdf6480b17 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:00.035525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1036/2096] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:51:59 2022] PUT /identity/v3/projects/6994396054cd464d8ac0668b3320a337/users/74c1232b400f4ccabe88a14199756686/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:00.039678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4aac8-d714-4a89-995d-009fdb5c5e51 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:00.051787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4aac8-d714-4a89-995d-009fdb5c5e51 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:00.115564 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31321a-ec9f-4f30-bddc-09128d0b2229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:00.116179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31321a-ec9f-4f30-bddc-09128d0b2229 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:00.116455 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31321a-ec9f-4f30-bddc-09128d0b2229 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:00.116727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31321a-ec9f-4f30-bddc-09128d0b2229 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:00.117479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31321a-ec9f-4f30-bddc-09128d0b2229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:00.120135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31321a-ec9f-4f30-bddc-09128d0b2229 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:00.135114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1061/2097] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:52:00.139044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd976b-6595-4755-8ddc-38bdf6480b17 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:00.149892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd976b-6595-4755-8ddc-38bdf6480b17 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:00.156314 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a08299-1fdb-469e-9df1-599a96983b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:00.157024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08299-1fdb-469e-9df1-599a96983b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:00.157281 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08299-1fdb-469e-9df1-599a96983b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:00.157579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08299-1fdb-469e-9df1-599a96983b17 admin admin] PATH_INFO: `/v3/projects/6994396054cd464d8ac0668b3320a337/users/74c1232b400f4ccabe88a14199756686/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:00.160004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a08299-1fdb-469e-9df1-599a96983b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6994396054cd464d8ac0668b3320a337, user_id=74c1232b400f4ccabe88a14199756686, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:00.162706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a08299-1fdb-469e-9df1-599a96983b17 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:00.175050 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1037/2098] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:52:00 2022] PUT /identity/v3/projects/6994396054cd464d8ac0668b3320a337/users/74c1232b400f4ccabe88a14199756686/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:00.187750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e1138-c40a-44d2-8b6f-2ff0cc02bd47 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:00.188232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e1138-c40a-44d2-8b6f-2ff0cc02bd47 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:00.188680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e1138-c40a-44d2-8b6f-2ff0cc02bd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:00.198941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:00.226490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-411e1138-c40a-44d2-8b6f-2ff0cc02bd47 None None] MFA Rules not processed for user `74c1232b400f4ccabe88a14199756686`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:00.229272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-411e1138-c40a-44d2-8b6f-2ff0cc02bd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:00.306783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1062/2099] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:00 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:00.317514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a08299-1fdb-469e-9df1-599a96983b17 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:00.319244 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-73a08299-1fdb-469e-9df1-599a96983b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:00.384225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a08299-1fdb-469e-9df1-599a96983b17 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:00.388768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0436e-64a3-42c6-97fd-38328d0236e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:00.389391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0436e-64a3-42c6-97fd-38328d0236e9 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:00.389669 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0436e-64a3-42c6-97fd-38328d0236e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:00.389942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0436e-64a3-42c6-97fd-38328d0236e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:00.403523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0436e-64a3-42c6-97fd-38328d0236e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:00.404526 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0436e-64a3-42c6-97fd-38328d0236e9 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:00.415131 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1038/2100] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:00 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:00.599222 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e1138-c40a-44d2-8b6f-2ff0cc02bd47 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:00.610099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e1138-c40a-44d2-8b6f-2ff0cc02bd47 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:00.613720 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce5058-833d-4948-955b-24e45ed27e7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:00.614337 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce5058-833d-4948-955b-24e45ed27e7d service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:00.614615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce5058-833d-4948-955b-24e45ed27e7d service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:00.614984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce5058-833d-4948-955b-24e45ed27e7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:00.678778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce5058-833d-4948-955b-24e45ed27e7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:00.680231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce5058-833d-4948-955b-24e45ed27e7d service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:00.691111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1063/2101] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:00 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:10.410274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=57,keystone:INSERT=5 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:10.686701 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=60 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:11.285694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0436e-64a3-42c6-97fd-38328d0236e9 service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:11.357999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0436e-64a3-42c6-97fd-38328d0236e9 service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:11.364952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:11.366046 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:11.366471 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:11.366954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] PATH_INFO: `/v3/users/3e52eeffb8cf424d885783c511d88183` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:11.368565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e52eeffb8cf424d885783c511d88183)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:11.371699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:11.433919 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e52eeffb8cf424d885783c511d88183'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:11.440850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e52eeffb8cf424d885783c511d88183'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:11.441544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e52eeffb8cf424d885783c511d88183'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:11.455619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e52eeffb8cf424d885783c511d88183'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:11.464834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e52eeffb8cf424d885783c511d88183'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:11.473101 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1039/2102] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:52:11 2022] DELETE /identity/v3/users/3e52eeffb8cf424d885783c511d88183 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:11.479336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31d7ee-b358-4530-804b-e4fc932bd854 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:11.479336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31d7ee-b358-4530-804b-e4fc932bd854 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:11.479336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31d7ee-b358-4530-804b-e4fc932bd854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:11.498345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-8b31d7ee-b358-4530-804b-e4fc932bd854 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:11.499607 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8b31d7ee-b358-4530-804b-e4fc932bd854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:11.565895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1064/2103] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:11.576494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:11.577916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:11.657932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71bfd1-7bc0-45d2-9ec4-e2fcd59737cc admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:11.661614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7949-bda3-4658-8c53-0815876b350b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:11.662234 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7949-bda3-4658-8c53-0815876b350b service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:11.662504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7949-bda3-4658-8c53-0815876b350b service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:11.662798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7949-bda3-4658-8c53-0815876b350b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:11.673581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba7949-bda3-4658-8c53-0815876b350b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:11.675053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba7949-bda3-4658-8c53-0815876b350b service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:11.686117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1040/2104] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:11.989192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31d7ee-b358-4530-804b-e4fc932bd854 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:11.990871 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8b31d7ee-b358-4530-804b-e4fc932bd854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:12.004987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31d7ee-b358-4530-804b-e4fc932bd854 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:12.013883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:12.014582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:12.014979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:12.015165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] PATH_INFO: `/v3/projects/06812155ebab467782d121c66d15496c` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:12.016387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06812155ebab467782d121c66d15496c)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:12.018418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:12.075520 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06812155ebab467782d121c66d15496c', 'request_id': 'req-9673fa05-bbe3-4834-b845-1ee331083856'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.082247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06812155ebab467782d121c66d15496c', 'request_id': 'req-9673fa05-bbe3-4834-b845-1ee331083856'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.092219 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06812155ebab467782d121c66d15496c', 'request_id': 'req-9673fa05-bbe3-4834-b845-1ee331083856'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.106964 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] The token cache is being invalidate because project 06812155ebab467782d121c66d15496c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:52:12.107252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.107919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.109452 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1065/2105] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:52:11 2022] DELETE /identity/v3/projects/06812155ebab467782d121c66d15496c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:12.113598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7949-bda3-4658-8c53-0815876b350b service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:12.123773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7949-bda3-4658-8c53-0815876b350b service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:12.142612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:12.186560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1dd59-9d47-44fb-b440-986e1a4ba8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:12.187208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1dd59-9d47-44fb-b440-986e1a4ba8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:12.187515 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1dd59-9d47-44fb-b440-986e1a4ba8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:12.187785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1dd59-9d47-44fb-b440-986e1a4ba8a5 admin admin] PATH_INFO: `/v3/users/74c1232b400f4ccabe88a14199756686` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:12.189019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1dd59-9d47-44fb-b440-986e1a4ba8a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74c1232b400f4ccabe88a14199756686)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:12.190974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1dd59-9d47-44fb-b440-986e1a4ba8a5 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:12.250524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-18a1dd59-9d47-44fb-b440-986e1a4ba8a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74c1232b400f4ccabe88a14199756686'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.258926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-18a1dd59-9d47-44fb-b440-986e1a4ba8a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74c1232b400f4ccabe88a14199756686'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.259879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-18a1dd59-9d47-44fb-b440-986e1a4ba8a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74c1232b400f4ccabe88a14199756686'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.276461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-18a1dd59-9d47-44fb-b440-986e1a4ba8a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74c1232b400f4ccabe88a14199756686'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.284502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-18a1dd59-9d47-44fb-b440-986e1a4ba8a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74c1232b400f4ccabe88a14199756686'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.291185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1041/2106] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:52:12 2022] DELETE /identity/v3/users/74c1232b400f4ccabe88a14199756686 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:52:12.619433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:12.620980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:12.648391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673fa05-bbe3-4834-b845-1ee331083856 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:12.745592 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c58ae-5346-4144-9c8b-baa9074e0426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:12.746456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c58ae-5346-4144-9c8b-baa9074e0426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:12.746791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c58ae-5346-4144-9c8b-baa9074e0426 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:12.747073 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c58ae-5346-4144-9c8b-baa9074e0426 admin admin] PATH_INFO: `/v3/projects/6994396054cd464d8ac0668b3320a337` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:12.748518 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c58ae-5346-4144-9c8b-baa9074e0426 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6994396054cd464d8ac0668b3320a337)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:12.750448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c58ae-5346-4144-9c8b-baa9074e0426 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:12.813471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:12.856283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-aa2c58ae-5346-4144-9c8b-baa9074e0426 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6994396054cd464d8ac0668b3320a337', 'request_id': 'req-aa2c58ae-5346-4144-9c8b-baa9074e0426'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.864055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-aa2c58ae-5346-4144-9c8b-baa9074e0426 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6994396054cd464d8ac0668b3320a337', 'request_id': 'req-aa2c58ae-5346-4144-9c8b-baa9074e0426'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.878305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-aa2c58ae-5346-4144-9c8b-baa9074e0426 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6994396054cd464d8ac0668b3320a337', 'request_id': 'req-aa2c58ae-5346-4144-9c8b-baa9074e0426'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.893904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-aa2c58ae-5346-4144-9c8b-baa9074e0426 admin admin] The token cache is being invalidate because project 6994396054cd464d8ac0668b3320a337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:52:12.894826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-aa2c58ae-5346-4144-9c8b-baa9074e0426 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.895695 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-aa2c58ae-5346-4144-9c8b-baa9074e0426 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:12.897597 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1066/2107] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:52:12 2022] DELETE /identity/v3/projects/6994396054cd464d8ac0668b3320a337 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:12.905117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de2cd9-48fb-4e5d-80e3-365c485c9b36 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:12.905706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de2cd9-48fb-4e5d-80e3-365c485c9b36 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:12.905706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de2cd9-48fb-4e5d-80e3-365c485c9b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:12.927230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-f6de2cd9-48fb-4e5d-80e3-365c485c9b36 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:12.928552 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f6de2cd9-48fb-4e5d-80e3-365c485c9b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:13.001674 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1042/2108] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:13.013400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54537a-2eb0-470a-b342-642b7e759949 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.013400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54537a-2eb0-470a-b342-642b7e759949 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.013400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54537a-2eb0-470a-b342-642b7e759949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:13.035774 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-eb54537a-2eb0-470a-b342-642b7e759949 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:13.036988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-eb54537a-2eb0-470a-b342-642b7e759949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:13.046065 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1067/2109] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:13.050334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de2cd9-48fb-4e5d-80e3-365c485c9b36 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:13.063768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de2cd9-48fb-4e5d-80e3-365c485c9b36 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:13.068581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126da541-2a46-4f6a-88d1-d302d655be0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:13.069196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126da541-2a46-4f6a-88d1-d302d655be0e admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.069502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126da541-2a46-4f6a-88d1-d302d655be0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.069819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126da541-2a46-4f6a-88d1-d302d655be0e admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:13.070666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126da541-2a46-4f6a-88d1-d302d655be0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:52:13.071034 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126da541-2a46-4f6a-88d1-d302d655be0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:13.073199 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126da541-2a46-4f6a-88d1-d302d655be0e admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:13.087747 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1043/2110] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:52:13.092278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54537a-2eb0-470a-b342-642b7e759949 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:13.102813 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54537a-2eb0-470a-b342-642b7e759949 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:13.108772 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e35c5a-7a1c-4db0-b35a-bbcc3e010fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:13.109721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e35c5a-7a1c-4db0-b35a-bbcc3e010fca admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.110046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e35c5a-7a1c-4db0-b35a-bbcc3e010fca admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.110327 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e35c5a-7a1c-4db0-b35a-bbcc3e010fca admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:13.111160 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e35c5a-7a1c-4db0-b35a-bbcc3e010fca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:13.113235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e35c5a-7a1c-4db0-b35a-bbcc3e010fca admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:13.149655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1068/2111] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:52:13 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:52:13.154551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126da541-2a46-4f6a-88d1-d302d655be0e admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:13.171385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126da541-2a46-4f6a-88d1-d302d655be0e admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:13.203520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:13.245979 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3911f3-3a59-4add-8373-80164e7be5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:13.246610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3911f3-3a59-4add-8373-80164e7be5b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.247116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3911f3-3a59-4add-8373-80164e7be5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.247462 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3911f3-3a59-4add-8373-80164e7be5b5 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:13.248206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3911f3-3a59-4add-8373-80164e7be5b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:13.250363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3911f3-3a59-4add-8373-80164e7be5b5 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:13.278884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1044/2112] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:52:13 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:52:13.280087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e35c5a-7a1c-4db0-b35a-bbcc3e010fca admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:13.293279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e35c5a-7a1c-4db0-b35a-bbcc3e010fca admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:13.298693 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd85d6-4585-46d8-a5e8-d7d68a4fafe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:13.299626 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd85d6-4585-46d8-a5e8-d7d68a4fafe4 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.299908 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd85d6-4585-46d8-a5e8-d7d68a4fafe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.300391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd85d6-4585-46d8-a5e8-d7d68a4fafe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:13.301099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd85d6-4585-46d8-a5e8-d7d68a4fafe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:13.302727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd85d6-4585-46d8-a5e8-d7d68a4fafe4 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:13.318774 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1069/2113] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:52:13.319984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3911f3-3a59-4add-8373-80164e7be5b5 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:13.336825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3911f3-3a59-4add-8373-80164e7be5b5 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:13.351098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9cc7b-744a-4406-a759-b170167ea25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:13.351737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9cc7b-744a-4406-a759-b170167ea25a admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.352073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9cc7b-744a-4406-a759-b170167ea25a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.352354 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9cc7b-744a-4406-a759-b170167ea25a admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:13.353058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9cc7b-744a-4406-a759-b170167ea25a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:13.354692 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9cc7b-744a-4406-a759-b170167ea25a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:13.370282 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1045/2114] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:13.371539 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd85d6-4585-46d8-a5e8-d7d68a4fafe4 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:13.384028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd85d6-4585-46d8-a5e8-d7d68a4fafe4 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:13.389381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da9fee-6325-4796-929e-6f07232f1c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:13.391187 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da9fee-6325-4796-929e-6f07232f1c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.391419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da9fee-6325-4796-929e-6f07232f1c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.391714 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da9fee-6325-4796-929e-6f07232f1c7a admin admin] PATH_INFO: `/v3/projects/bf475e3f30024003ad09d16bc2ff081e/users/8a0bd860c59249ad93a660659d06a44d/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:13.409709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da9fee-6325-4796-929e-6f07232f1c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf475e3f30024003ad09d16bc2ff081e, user_id=8a0bd860c59249ad93a660659d06a44d, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:13.412444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da9fee-6325-4796-929e-6f07232f1c7a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:13.434755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1070/2115] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:52:13 2022] PUT /identity/v3/projects/bf475e3f30024003ad09d16bc2ff081e/users/8a0bd860c59249ad93a660659d06a44d/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:52:13.443978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482ca57-c324-4361-96ea-f2aeba5c6e45 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.444302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482ca57-c324-4361-96ea-f2aeba5c6e45 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.444571 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482ca57-c324-4361-96ea-f2aeba5c6e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:13.481010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-f482ca57-c324-4361-96ea-f2aeba5c6e45 None None] MFA Rules not processed for user `8a0bd860c59249ad93a660659d06a44d`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:13.482410 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f482ca57-c324-4361-96ea-f2aeba5c6e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:13.578856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1046/2116] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:13.583304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da9fee-6325-4796-929e-6f07232f1c7a admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:13.671358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da9fee-6325-4796-929e-6f07232f1c7a admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:13.676468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16509dc3-2f9a-4f4f-a988-a9a126496edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:13.677083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16509dc3-2f9a-4f4f-a988-a9a126496edb admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.677483 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16509dc3-2f9a-4f4f-a988-a9a126496edb admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.677841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16509dc3-2f9a-4f4f-a988-a9a126496edb admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:13.678617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16509dc3-2f9a-4f4f-a988-a9a126496edb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:13.680978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16509dc3-2f9a-4f4f-a988-a9a126496edb admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:13.710922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1071/2117] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:52:13 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:52:13.716139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482ca57-c324-4361-96ea-f2aeba5c6e45 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:13.727430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482ca57-c324-4361-96ea-f2aeba5c6e45 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:13.803280 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1359c-024b-4285-8c74-a916c50530eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:13.803963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1359c-024b-4285-8c74-a916c50530eb admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.804340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1359c-024b-4285-8c74-a916c50530eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.804700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1359c-024b-4285-8c74-a916c50530eb admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:13.805991 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1359c-024b-4285-8c74-a916c50530eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:13.808063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1359c-024b-4285-8c74-a916c50530eb admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:13.830648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1047/2118] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:52:13 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:52:13.834993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16509dc3-2f9a-4f4f-a988-a9a126496edb admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:13.846491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16509dc3-2f9a-4f4f-a988-a9a126496edb admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:13.851566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbb300-364b-4fb6-bce0-9276ae8f9aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:13.852359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbb300-364b-4fb6-bce0-9276ae8f9aad admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.852640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbb300-364b-4fb6-bce0-9276ae8f9aad admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.852909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbb300-364b-4fb6-bce0-9276ae8f9aad admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:13.853606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbb300-364b-4fb6-bce0-9276ae8f9aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:13.855256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbb300-364b-4fb6-bce0-9276ae8f9aad admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:13.866587 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1072/2119] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:13.871651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1359c-024b-4285-8c74-a916c50530eb admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:13.885308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1359c-024b-4285-8c74-a916c50530eb admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:13.890077 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159f47c-0eab-4a2b-b86d-7b81cbcca31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:13.890726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159f47c-0eab-4a2b-b86d-7b81cbcca31e admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.891007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159f47c-0eab-4a2b-b86d-7b81cbcca31e admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.891430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159f47c-0eab-4a2b-b86d-7b81cbcca31e admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:13.892183 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159f47c-0eab-4a2b-b86d-7b81cbcca31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:13.893790 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159f47c-0eab-4a2b-b86d-7b81cbcca31e admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:13.905993 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1048/2120] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:13.910194 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbb300-364b-4fb6-bce0-9276ae8f9aad admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:13.920925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbb300-364b-4fb6-bce0-9276ae8f9aad admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:13.925903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9238ce-8a45-4c30-8354-5c4d9d883f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:13.927428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9238ce-8a45-4c30-8354-5c4d9d883f6a admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.927428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9238ce-8a45-4c30-8354-5c4d9d883f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.927428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9238ce-8a45-4c30-8354-5c4d9d883f6a admin admin] PATH_INFO: `/v3/projects/48985d87c8e34109b31d4b619213f349/users/bf73808cf84443a1a82d2de9a18d5af1/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:13.947389 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9238ce-8a45-4c30-8354-5c4d9d883f6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=48985d87c8e34109b31d4b619213f349, user_id=bf73808cf84443a1a82d2de9a18d5af1, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:13.950150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9238ce-8a45-4c30-8354-5c4d9d883f6a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:13.962676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:13.967966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1073/2121] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:52:13 2022] PUT /identity/v3/projects/48985d87c8e34109b31d4b619213f349/users/bf73808cf84443a1a82d2de9a18d5af1/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:13.979834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f28db-aff1-48e5-9b12-8bd93a6fdfc0 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:13.980121 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f28db-aff1-48e5-9b12-8bd93a6fdfc0 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:13.980396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f28db-aff1-48e5-9b12-8bd93a6fdfc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:14.015879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-6c7f28db-aff1-48e5-9b12-8bd93a6fdfc0 None None] MFA Rules not processed for user `bf73808cf84443a1a82d2de9a18d5af1`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:14.017104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6c7f28db-aff1-48e5-9b12-8bd93a6fdfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:14.021333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:14.094806 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1049/2122] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:14.099838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9238ce-8a45-4c30-8354-5c4d9d883f6a admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:14.173213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9238ce-8a45-4c30-8354-5c4d9d883f6a admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:14.178220 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c423252-7c08-4243-98fa-24f726e60e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:14.178899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423252-7c08-4243-98fa-24f726e60e10 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:14.179237 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423252-7c08-4243-98fa-24f726e60e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:14.179534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c423252-7c08-4243-98fa-24f726e60e10 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:14.181136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c423252-7c08-4243-98fa-24f726e60e10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:14.183491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c423252-7c08-4243-98fa-24f726e60e10 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:14.212776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1074/2123] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:52:14 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:52:14.231025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f28db-aff1-48e5-9b12-8bd93a6fdfc0 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:14.231025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f28db-aff1-48e5-9b12-8bd93a6fdfc0 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:14.301356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527c184-ecb0-4c10-8aa4-5aa139708c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:14.302073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527c184-ecb0-4c10-8aa4-5aa139708c5f admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:14.302434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527c184-ecb0-4c10-8aa4-5aa139708c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:14.302854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527c184-ecb0-4c10-8aa4-5aa139708c5f admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:14.303567 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527c184-ecb0-4c10-8aa4-5aa139708c5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:14.306565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527c184-ecb0-4c10-8aa4-5aa139708c5f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:14.331321 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1050/2124] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:52:14 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:52:14.336024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c423252-7c08-4243-98fa-24f726e60e10 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:14.350761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c423252-7c08-4243-98fa-24f726e60e10 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:14.357022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7067c72-9626-4305-80b8-eecc2216b8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:14.357667 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7067c72-9626-4305-80b8-eecc2216b8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:14.357947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7067c72-9626-4305-80b8-eecc2216b8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:14.358225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7067c72-9626-4305-80b8-eecc2216b8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:14.358994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7067c72-9626-4305-80b8-eecc2216b8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:14.360657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7067c72-9626-4305-80b8-eecc2216b8a8 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:14.374053 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1075/2125] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:52:14.378302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527c184-ecb0-4c10-8aa4-5aa139708c5f admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:14.389812 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527c184-ecb0-4c10-8aa4-5aa139708c5f admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:14.398947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66c986-3587-4f9c-8e55-e78e9901481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:14.400165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66c986-3587-4f9c-8e55-e78e9901481d admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:14.400811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66c986-3587-4f9c-8e55-e78e9901481d admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:14.401293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66c986-3587-4f9c-8e55-e78e9901481d admin admin] PATH_INFO: `/v3/projects/88099b4dabb94431947fff320f7e7c61/users/66ca07f7bfe349fdb01e1e50c4354c89/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:14.421311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66c986-3587-4f9c-8e55-e78e9901481d admin admin] RBAC: Authorizing `identity:create_grant(project_id=88099b4dabb94431947fff320f7e7c61, user_id=66ca07f7bfe349fdb01e1e50c4354c89, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:14.434694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66c986-3587-4f9c-8e55-e78e9901481d admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:14.457619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1051/2126] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:52:14 2022] PUT /identity/v3/projects/88099b4dabb94431947fff320f7e7c61/users/66ca07f7bfe349fdb01e1e50c4354c89/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:52:14.461605 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7067c72-9626-4305-80b8-eecc2216b8a8 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:14.485675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7067c72-9626-4305-80b8-eecc2216b8a8 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:14.592458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b761eed-96d1-4355-8f53-d80c5be17725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:14.593566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b761eed-96d1-4355-8f53-d80c5be17725 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:14.594019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b761eed-96d1-4355-8f53-d80c5be17725 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:14.594677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b761eed-96d1-4355-8f53-d80c5be17725 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:14.595866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b761eed-96d1-4355-8f53-d80c5be17725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:14.597722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b761eed-96d1-4355-8f53-d80c5be17725 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:14.617341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1076/2127] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:14.620105 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66c986-3587-4f9c-8e55-e78e9901481d admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:14.637695 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66c986-3587-4f9c-8e55-e78e9901481d admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:14.644418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8006a4-6ef8-4e19-8228-04bb93bdffff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:14.645344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8006a4-6ef8-4e19-8228-04bb93bdffff admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:14.646040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8006a4-6ef8-4e19-8228-04bb93bdffff admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:14.646361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8006a4-6ef8-4e19-8228-04bb93bdffff admin admin] PATH_INFO: `/v3/projects/88099b4dabb94431947fff320f7e7c61/users/66ca07f7bfe349fdb01e1e50c4354c89/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:14.649646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8006a4-6ef8-4e19-8228-04bb93bdffff admin admin] RBAC: Authorizing `identity:create_grant(project_id=88099b4dabb94431947fff320f7e7c61, user_id=66ca07f7bfe349fdb01e1e50c4354c89, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:14.652578 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8006a4-6ef8-4e19-8228-04bb93bdffff admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:14.664855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1052/2128] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:52:14 2022] PUT /identity/v3/projects/88099b4dabb94431947fff320f7e7c61/users/66ca07f7bfe349fdb01e1e50c4354c89/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:14.678283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409f98d-5460-4481-8739-33f145261816 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:14.678603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409f98d-5460-4481-8739-33f145261816 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:14.679033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409f98d-5460-4481-8739-33f145261816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:14.713387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-4409f98d-5460-4481-8739-33f145261816 None None] MFA Rules not processed for user `66ca07f7bfe349fdb01e1e50c4354c89`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:14.723626 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4409f98d-5460-4481-8739-33f145261816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:14.818473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:14.833268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1077/2129] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:14.929895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8006a4-6ef8-4e19-8228-04bb93bdffff admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:14.932811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3c8006a4-6ef8-4e19-8228-04bb93bdffff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:14.998330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8006a4-6ef8-4e19-8228-04bb93bdffff admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:15.002839 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df3889-8d1e-40e2-b646-801c267fe3d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:15.003519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df3889-8d1e-40e2-b646-801c267fe3d5 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:15.003816 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df3889-8d1e-40e2-b646-801c267fe3d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:15.004090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df3889-8d1e-40e2-b646-801c267fe3d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:15.014589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df3889-8d1e-40e2-b646-801c267fe3d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:15.016120 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df3889-8d1e-40e2-b646-801c267fe3d5 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:15.021354 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:15.028885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1053/2130] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:15.080173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409f98d-5460-4481-8739-33f145261816 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:15.093591 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409f98d-5460-4481-8739-33f145261816 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:15.098110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467b43b-1192-4b3d-88e8-c0934a1f5b45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:15.099846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467b43b-1192-4b3d-88e8-c0934a1f5b45 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:15.100152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467b43b-1192-4b3d-88e8-c0934a1f5b45 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:15.100787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467b43b-1192-4b3d-88e8-c0934a1f5b45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:15.171589 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467b43b-1192-4b3d-88e8-c0934a1f5b45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:15.173225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467b43b-1192-4b3d-88e8-c0934a1f5b45 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:15.185633 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1078/2131] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:21.469673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df3889-8d1e-40e2-b646-801c267fe3d5 service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:21.545321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467b43b-1192-4b3d-88e8-c0934a1f5b45 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:21.546969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9467b43b-1192-4b3d-88e8-c0934a1f5b45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:21.578192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df3889-8d1e-40e2-b646-801c267fe3d5 service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:21.584242 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39498b2-b26d-4164-83ab-a437d3bc47c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:21.584913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39498b2-b26d-4164-83ab-a437d3bc47c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:21.585188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39498b2-b26d-4164-83ab-a437d3bc47c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:21.585464 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39498b2-b26d-4164-83ab-a437d3bc47c7 admin admin] PATH_INFO: `/v3/users/8a0bd860c59249ad93a660659d06a44d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:21.587813 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39498b2-b26d-4164-83ab-a437d3bc47c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a0bd860c59249ad93a660659d06a44d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:21.591737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39498b2-b26d-4164-83ab-a437d3bc47c7 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:21.644606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467b43b-1192-4b3d-88e8-c0934a1f5b45 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:21.650199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3781b6d-d486-4637-8f14-4037a4b99a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:21.650788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3781b6d-d486-4637-8f14-4037a4b99a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:21.651065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b39498b2-b26d-4164-83ab-a437d3bc47c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a0bd860c59249ad93a660659d06a44d'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:21.651180 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3781b6d-d486-4637-8f14-4037a4b99a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:21.651329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3781b6d-d486-4637-8f14-4037a4b99a45 admin admin] PATH_INFO: `/v3/users/6b8e17d8c3d94e6fb28191c5988f8c6a` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:21.652502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3781b6d-d486-4637-8f14-4037a4b99a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b8e17d8c3d94e6fb28191c5988f8c6a)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:21.654481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3781b6d-d486-4637-8f14-4037a4b99a45 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:21.659988 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b39498b2-b26d-4164-83ab-a437d3bc47c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a0bd860c59249ad93a660659d06a44d'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:21.660740 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b39498b2-b26d-4164-83ab-a437d3bc47c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a0bd860c59249ad93a660659d06a44d'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:21.673101 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b39498b2-b26d-4164-83ab-a437d3bc47c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a0bd860c59249ad93a660659d06a44d'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:21.685591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b39498b2-b26d-4164-83ab-a437d3bc47c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a0bd860c59249ad93a660659d06a44d'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:21.695637 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1054/2132] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:52:21 2022] DELETE /identity/v3/users/8a0bd860c59249ad93a660659d06a44d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:21.701970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93e9f9-3b1a-424b-bfe2-699116cd28ea None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:21.702247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93e9f9-3b1a-424b-bfe2-699116cd28ea None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:21.702618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93e9f9-3b1a-424b-bfe2-699116cd28ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:21.717704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f3781b6d-d486-4637-8f14-4037a4b99a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b8e17d8c3d94e6fb28191c5988f8c6a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:21.724862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-dd93e9f9-3b1a-424b-bfe2-699116cd28ea None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:21.726088 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-dd93e9f9-3b1a-424b-bfe2-699116cd28ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:21.732411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f3781b6d-d486-4637-8f14-4037a4b99a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b8e17d8c3d94e6fb28191c5988f8c6a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:21.741740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f3781b6d-d486-4637-8f14-4037a4b99a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b8e17d8c3d94e6fb28191c5988f8c6a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:21.742473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f3781b6d-d486-4637-8f14-4037a4b99a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b8e17d8c3d94e6fb28191c5988f8c6a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:21.744298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f3781b6d-d486-4637-8f14-4037a4b99a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b8e17d8c3d94e6fb28191c5988f8c6a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:21.766677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1079/2133] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:52:21 2022] DELETE /identity/v3/users/6b8e17d8c3d94e6fb28191c5988f8c6a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:52:21.767612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0fc31-4d39-46cc-9f19-c944ee2a8795 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:21.767894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0fc31-4d39-46cc-9f19-c944ee2a8795 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:21.768167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0fc31-4d39-46cc-9f19-c944ee2a8795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:21.790930 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-6ef0fc31-4d39-46cc-9f19-c944ee2a8795 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:21.792165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6ef0fc31-4d39-46cc-9f19-c944ee2a8795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:21.802534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:21.815721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1055/2134] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:52:21.823997 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93e9f9-3b1a-424b-bfe2-699116cd28ea None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:21.826452 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-dd93e9f9-3b1a-424b-bfe2-699116cd28ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:21.865831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:21.888458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1080/2135] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:52:21.897984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0fc31-4d39-46cc-9f19-c944ee2a8795 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:21.899444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6ef0fc31-4d39-46cc-9f19-c944ee2a8795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:21.926925 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93e9f9-3b1a-424b-bfe2-699116cd28ea None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:21.978389 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0fc31-4d39-46cc-9f19-c944ee2a8795 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:21.982675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f196866-5b62-4c07-b32f-1d57a9086cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:21.983629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f196866-5b62-4c07-b32f-1d57a9086cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:21.983903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f196866-5b62-4c07-b32f-1d57a9086cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:21.984172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f196866-5b62-4c07-b32f-1d57a9086cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:21.996841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f196866-5b62-4c07-b32f-1d57a9086cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:21.998419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f196866-5b62-4c07-b32f-1d57a9086cf0 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:22.007853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856c208-4f30-48b0-b901-64e6b2bb81cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:22.009434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856c208-4f30-48b0-b901-64e6b2bb81cc service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:22.009749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856c208-4f30-48b0-b901-64e6b2bb81cc service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:22.010031 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856c208-4f30-48b0-b901-64e6b2bb81cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:22.011362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1081/2136] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:52:22.026549 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856c208-4f30-48b0-b901-64e6b2bb81cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:22.028062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856c208-4f30-48b0-b901-64e6b2bb81cc service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:22.042455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1056/2137] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:52:22.471047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f196866-5b62-4c07-b32f-1d57a9086cf0 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:22.472630 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3f196866-5b62-4c07-b32f-1d57a9086cf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:22.484186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856c208-4f30-48b0-b901-64e6b2bb81cc service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:22.485610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5856c208-4f30-48b0-b901-64e6b2bb81cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:22.493426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f196866-5b62-4c07-b32f-1d57a9086cf0 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:22.499986 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:22.500931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:22.501409 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856c208-4f30-48b0-b901-64e6b2bb81cc service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:22.501662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:22.502138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] PATH_INFO: `/v3/projects/bf475e3f30024003ad09d16bc2ff081e` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:22.503757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf475e3f30024003ad09d16bc2ff081e)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:22.506184 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:22.507351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8527a3-4ff5-4355-99fd-61f69611573c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:22.508077 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8527a3-4ff5-4355-99fd-61f69611573c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:22.508543 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8527a3-4ff5-4355-99fd-61f69611573c admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:22.508980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8527a3-4ff5-4355-99fd-61f69611573c admin admin] PATH_INFO: `/v3/projects/5c161b21fd204792884cb02e3d895c99` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:22.510347 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8527a3-4ff5-4355-99fd-61f69611573c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c161b21fd204792884cb02e3d895c99)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:22.512527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8527a3-4ff5-4355-99fd-61f69611573c admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:22.571468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7d8527a3-4ff5-4355-99fd-61f69611573c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c161b21fd204792884cb02e3d895c99', 'request_id': 'req-7d8527a3-4ff5-4355-99fd-61f69611573c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.574247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf475e3f30024003ad09d16bc2ff081e', 'request_id': 'req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.579839 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7d8527a3-4ff5-4355-99fd-61f69611573c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c161b21fd204792884cb02e3d895c99', 'request_id': 'req-7d8527a3-4ff5-4355-99fd-61f69611573c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.581154 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf475e3f30024003ad09d16bc2ff081e', 'request_id': 'req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.589517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7d8527a3-4ff5-4355-99fd-61f69611573c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c161b21fd204792884cb02e3d895c99', 'request_id': 'req-7d8527a3-4ff5-4355-99fd-61f69611573c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.592367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf475e3f30024003ad09d16bc2ff081e', 'request_id': 'req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.601646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7d8527a3-4ff5-4355-99fd-61f69611573c admin admin] The token cache is being invalidate because project 5c161b21fd204792884cb02e3d895c99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:52:22.601968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7d8527a3-4ff5-4355-99fd-61f69611573c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.602945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7d8527a3-4ff5-4355-99fd-61f69611573c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.604490 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1057/2138] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:52:22 2022] DELETE /identity/v3/projects/5c161b21fd204792884cb02e3d895c99 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:22.606655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] The token cache is being invalidate because project bf475e3f30024003ad09d16bc2ff081e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:52:22.606946 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.607588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.609391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738938d-660e-4c89-8076-bf52f3de9bbe None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:22.610768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1082/2139] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:52:22 2022] DELETE /identity/v3/projects/bf475e3f30024003ad09d16bc2ff081e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:22.610883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738938d-660e-4c89-8076-bf52f3de9bbe None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:22.610883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738938d-660e-4c89-8076-bf52f3de9bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:22.613091 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:22.614810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:22.631216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-9738938d-660e-4c89-8076-bf52f3de9bbe None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:22.632431 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9738938d-660e-4c89-8076-bf52f3de9bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:22.676405 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:22.713584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d95dfa-bab8-4f18-8efa-7234c02ec1e2 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:22.716117 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1058/2140] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:22.722918 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2733b-5939-4594-9737-e5182f04860c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:22.723731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2733b-5939-4594-9737-e5182f04860c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:22.724010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2733b-5939-4594-9737-e5182f04860c admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:22.724218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c59ac-7988-48aa-9b58-d048f82a7a7a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:22.724344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb2733b-5939-4594-9737-e5182f04860c admin admin] PATH_INFO: `/v3/users/bf73808cf84443a1a82d2de9a18d5af1` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:22.724503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c59ac-7988-48aa-9b58-d048f82a7a7a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:22.724770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c59ac-7988-48aa-9b58-d048f82a7a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:22.725507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2733b-5939-4594-9737-e5182f04860c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf73808cf84443a1a82d2de9a18d5af1)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:22.727799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb2733b-5939-4594-9737-e5182f04860c admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:22.735544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:22.754218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-9c9c59ac-7988-48aa-9b58-d048f82a7a7a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:22.756843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9c9c59ac-7988-48aa-9b58-d048f82a7a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:22.773641 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1059/2141] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:52:22.780365 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c59ac-7988-48aa-9b58-d048f82a7a7a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:22.788693 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4bb2733b-5939-4594-9737-e5182f04860c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf73808cf84443a1a82d2de9a18d5af1'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.791909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c59ac-7988-48aa-9b58-d048f82a7a7a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:22.796831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074573a-f29b-45a4-b7ce-40138e0a853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:22.797447 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074573a-f29b-45a4-b7ce-40138e0a853a admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:22.797719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074573a-f29b-45a4-b7ce-40138e0a853a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:22.797984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074573a-f29b-45a4-b7ce-40138e0a853a admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:22.798719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074573a-f29b-45a4-b7ce-40138e0a853a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:52:22.799024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074573a-f29b-45a4-b7ce-40138e0a853a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:22.800648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074573a-f29b-45a4-b7ce-40138e0a853a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:22.801899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4bb2733b-5939-4594-9737-e5182f04860c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf73808cf84443a1a82d2de9a18d5af1'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.810703 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4bb2733b-5939-4594-9737-e5182f04860c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf73808cf84443a1a82d2de9a18d5af1'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.811452 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4bb2733b-5939-4594-9737-e5182f04860c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf73808cf84443a1a82d2de9a18d5af1'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.812498 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4bb2733b-5939-4594-9737-e5182f04860c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf73808cf84443a1a82d2de9a18d5af1'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:22.818829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1060/2142] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:52:22.820196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074573a-f29b-45a4-b7ce-40138e0a853a admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:22.822688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b074573a-f29b-45a4-b7ce-40138e0a853a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:22.828094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1083/2143] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:52:22 2022] DELETE /identity/v3/users/bf73808cf84443a1a82d2de9a18d5af1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:52:22.840857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074573a-f29b-45a4-b7ce-40138e0a853a admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:22.908490 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aeb022-4071-4a90-a71b-3394ffc8fc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:22.909130 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aeb022-4071-4a90-a71b-3394ffc8fc5b admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:22.909415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aeb022-4071-4a90-a71b-3394ffc8fc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:22.909685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aeb022-4071-4a90-a71b-3394ffc8fc5b admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:22.910354 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aeb022-4071-4a90-a71b-3394ffc8fc5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:22.912320 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aeb022-4071-4a90-a71b-3394ffc8fc5b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:22.944637 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1061/2144] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:52:22 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:52:22.949354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2733b-5939-4594-9737-e5182f04860c admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:22.961018 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb2733b-5939-4594-9737-e5182f04860c admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:23.041314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeb4d8-df19-458a-83c9-1f67c11be4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:23.041997 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeb4d8-df19-458a-83c9-1f67c11be4c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:23.042275 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeb4d8-df19-458a-83c9-1f67c11be4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:23.042544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeb4d8-df19-458a-83c9-1f67c11be4c2 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:23.043551 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daeb4d8-df19-458a-83c9-1f67c11be4c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:23.046174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daeb4d8-df19-458a-83c9-1f67c11be4c2 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:23.079274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1084/2145] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:52:22 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:52:23.083914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aeb022-4071-4a90-a71b-3394ffc8fc5b admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:23.096018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aeb022-4071-4a90-a71b-3394ffc8fc5b admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:23.103747 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469afb6-ca2d-481c-9277-6755d9578980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:23.104590 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469afb6-ca2d-481c-9277-6755d9578980 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:23.105109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469afb6-ca2d-481c-9277-6755d9578980 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:23.105466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469afb6-ca2d-481c-9277-6755d9578980 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:23.106294 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469afb6-ca2d-481c-9277-6755d9578980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:23.108067 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469afb6-ca2d-481c-9277-6755d9578980 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:23.123428 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1062/2146] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:23.126726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeb4d8-df19-458a-83c9-1f67c11be4c2 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:23.141006 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeb4d8-df19-458a-83c9-1f67c11be4c2 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:23.146569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc23501-9dd2-41d3-b239-d195f09e08fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:23.147230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc23501-9dd2-41d3-b239-d195f09e08fd admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:23.147541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc23501-9dd2-41d3-b239-d195f09e08fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:23.147776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc23501-9dd2-41d3-b239-d195f09e08fd admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:23.148513 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc23501-9dd2-41d3-b239-d195f09e08fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:23.151025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc23501-9dd2-41d3-b239-d195f09e08fd admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:23.167952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1085/2147] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:23.168302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469afb6-ca2d-481c-9277-6755d9578980 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:23.176636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc23501-9dd2-41d3-b239-d195f09e08fd admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:23.181066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3cc23501-9dd2-41d3-b239-d195f09e08fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:23.184923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469afb6-ca2d-481c-9277-6755d9578980 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:23.191799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a18121-eb8d-4bcc-add1-862f8afdb6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:23.194055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a18121-eb8d-4bcc-add1-862f8afdb6c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:23.194605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a18121-eb8d-4bcc-add1-862f8afdb6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:23.196651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a18121-eb8d-4bcc-add1-862f8afdb6c0 admin admin] PATH_INFO: `/v3/projects/018e12e98f1143d6acaf348cfea45763/users/acfba24ac674454d86e415b38c0c4618/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:23.205754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc23501-9dd2-41d3-b239-d195f09e08fd admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:23.215763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:23.216497 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:23.216779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:23.217052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] PATH_INFO: `/v3/projects/48985d87c8e34109b31d4b619213f349` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:23.218307 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] RBAC: Authorizing `identity:delete_project(project_id=48985d87c8e34109b31d4b619213f349)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:23.220325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:23.248055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a18121-eb8d-4bcc-add1-862f8afdb6c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=018e12e98f1143d6acaf348cfea45763, user_id=acfba24ac674454d86e415b38c0c4618, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:23.252567 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a18121-eb8d-4bcc-add1-862f8afdb6c0 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:23.278679 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1063/2148] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:52:23 2022] PUT /identity/v3/projects/018e12e98f1143d6acaf348cfea45763/users/acfba24ac674454d86e415b38c0c4618/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:52:23.280369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:23.283451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a771-b341-410d-a68d-c8c6be8553c7 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:23.283907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a771-b341-410d-a68d-c8c6be8553c7 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:23.284334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a771-b341-410d-a68d-c8c6be8553c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:23.296818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48985d87c8e34109b31d4b619213f349', 'request_id': 'req-a152ead0-0541-4804-bb69-58402033990b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:23.306452 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48985d87c8e34109b31d4b619213f349', 'request_id': 'req-a152ead0-0541-4804-bb69-58402033990b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:23.313007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-1b45a771-b341-410d-a68d-c8c6be8553c7 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:23.314498 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-1b45a771-b341-410d-a68d-c8c6be8553c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:23.316330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48985d87c8e34109b31d4b619213f349', 'request_id': 'req-a152ead0-0541-4804-bb69-58402033990b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:23.331202 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] The token cache is being invalidate because project 48985d87c8e34109b31d4b619213f349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:52:23.331635 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:23.332318 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:23.334777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1086/2149] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:52:23 2022] DELETE /identity/v3/projects/48985d87c8e34109b31d4b619213f349 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:52:23.338800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:23.341172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:23.409543 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1064/2150] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:52:23.421377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45a771-b341-410d-a68d-c8c6be8553c7 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:23.421744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152ead0-0541-4804-bb69-58402033990b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:23.422967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-1b45a771-b341-410d-a68d-c8c6be8553c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:23.426376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa21fc2-d1ae-4e80-8494-94cfe0c9fe40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:23.427056 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa21fc2-d1ae-4e80-8494-94cfe0c9fe40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:23.427335 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa21fc2-d1ae-4e80-8494-94cfe0c9fe40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:23.427606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa21fc2-d1ae-4e80-8494-94cfe0c9fe40 admin admin] PATH_INFO: `/v3/users/66ca07f7bfe349fdb01e1e50c4354c89` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:23.428810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa21fc2-d1ae-4e80-8494-94cfe0c9fe40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ca07f7bfe349fdb01e1e50c4354c89)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:23.431729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa21fc2-d1ae-4e80-8494-94cfe0c9fe40 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:23.462873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:23.493863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45a771-b341-410d-a68d-c8c6be8553c7 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:23.498579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ddf0e-883c-47ab-a45d-a8366e3ec6cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:23.499318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ddf0e-883c-47ab-a45d-a8366e3ec6cf service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:23.499814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ddf0e-883c-47ab-a45d-a8366e3ec6cf service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:23.500148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ddf0e-883c-47ab-a45d-a8366e3ec6cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:23.500703 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-aaa21fc2-d1ae-4e80-8494-94cfe0c9fe40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ca07f7bfe349fdb01e1e50c4354c89'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:23.519096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ddf0e-883c-47ab-a45d-a8366e3ec6cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:23.520781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ddf0e-883c-47ab-a45d-a8366e3ec6cf service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:23.523642 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-aaa21fc2-d1ae-4e80-8494-94cfe0c9fe40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ca07f7bfe349fdb01e1e50c4354c89'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:23.533335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1065/2151] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:52:23.534699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-aaa21fc2-d1ae-4e80-8494-94cfe0c9fe40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ca07f7bfe349fdb01e1e50c4354c89'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:23.535328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-aaa21fc2-d1ae-4e80-8494-94cfe0c9fe40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ca07f7bfe349fdb01e1e50c4354c89'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:23.536132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-aaa21fc2-d1ae-4e80-8494-94cfe0c9fe40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ca07f7bfe349fdb01e1e50c4354c89'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:23.554466 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1087/2152] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:52:23 2022] DELETE /identity/v3/users/66ca07f7bfe349fdb01e1e50c4354c89 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:23.907871 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ddf0e-883c-47ab-a45d-a8366e3ec6cf service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:23.921913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ddf0e-883c-47ab-a45d-a8366e3ec6cf service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:24.019081 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72554144-0bc0-42dd-b0f1-6285beadce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:24.019081 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72554144-0bc0-42dd-b0f1-6285beadce77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:24.019515 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72554144-0bc0-42dd-b0f1-6285beadce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:24.020275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72554144-0bc0-42dd-b0f1-6285beadce77 admin admin] PATH_INFO: `/v3/projects/88099b4dabb94431947fff320f7e7c61` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:24.021662 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72554144-0bc0-42dd-b0f1-6285beadce77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88099b4dabb94431947fff320f7e7c61)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:24.025130 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72554144-0bc0-42dd-b0f1-6285beadce77 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:24.099660 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-72554144-0bc0-42dd-b0f1-6285beadce77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88099b4dabb94431947fff320f7e7c61', 'request_id': 'req-72554144-0bc0-42dd-b0f1-6285beadce77'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:24.111303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-72554144-0bc0-42dd-b0f1-6285beadce77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88099b4dabb94431947fff320f7e7c61', 'request_id': 'req-72554144-0bc0-42dd-b0f1-6285beadce77'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:24.124401 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-72554144-0bc0-42dd-b0f1-6285beadce77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88099b4dabb94431947fff320f7e7c61', 'request_id': 'req-72554144-0bc0-42dd-b0f1-6285beadce77'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:24.140482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-72554144-0bc0-42dd-b0f1-6285beadce77 admin admin] The token cache is being invalidate because project 88099b4dabb94431947fff320f7e7c61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:52:24.140806 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-72554144-0bc0-42dd-b0f1-6285beadce77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:24.141496 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-72554144-0bc0-42dd-b0f1-6285beadce77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:24.144423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1066/2153] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:52:23 2022] DELETE /identity/v3/projects/88099b4dabb94431947fff320f7e7c61 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:24.151181 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b7c5-3dde-4a03-8e4e-8d276f168d8c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:24.152120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b7c5-3dde-4a03-8e4e-8d276f168d8c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:24.152120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b7c5-3dde-4a03-8e4e-8d276f168d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:24.175851 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-a123b7c5-3dde-4a03-8e4e-8d276f168d8c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:24.177088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a123b7c5-3dde-4a03-8e4e-8d276f168d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:24.217978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:24.253122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1088/2154] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:24.263093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403c4cf-6d47-4d64-93aa-a6fa54769f13 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:24.263519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403c4cf-6d47-4d64-93aa-a6fa54769f13 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:24.263886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403c4cf-6d47-4d64-93aa-a6fa54769f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:24.292809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-2403c4cf-6d47-4d64-93aa-a6fa54769f13 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:24.294565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2403c4cf-6d47-4d64-93aa-a6fa54769f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:24.308264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1067/2155] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:24.310263 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123b7c5-3dde-4a03-8e4e-8d276f168d8c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:24.324603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123b7c5-3dde-4a03-8e4e-8d276f168d8c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:24.343215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08320150-995c-4ae5-b458-239de2ab5b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:24.344498 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08320150-995c-4ae5-b458-239de2ab5b4d admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:24.344783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08320150-995c-4ae5-b458-239de2ab5b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:24.345053 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08320150-995c-4ae5-b458-239de2ab5b4d admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:24.345766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08320150-995c-4ae5-b458-239de2ab5b4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:52:24.346142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08320150-995c-4ae5-b458-239de2ab5b4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:24.347866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08320150-995c-4ae5-b458-239de2ab5b4d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:24.370862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1089/2156] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:52:24.375214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403c4cf-6d47-4d64-93aa-a6fa54769f13 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:24.385537 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403c4cf-6d47-4d64-93aa-a6fa54769f13 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:24.390443 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840abef-429e-4d86-9375-7854026b01c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:24.391146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840abef-429e-4d86-9375-7854026b01c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:24.391419 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840abef-429e-4d86-9375-7854026b01c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:24.391683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840abef-429e-4d86-9375-7854026b01c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:24.392597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840abef-429e-4d86-9375-7854026b01c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:24.394250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840abef-429e-4d86-9375-7854026b01c6 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:24.466931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1068/2157] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:52:24 2022] POST /identity/v3/projects => generated 383 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:52:24.467630 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08320150-995c-4ae5-b458-239de2ab5b4d admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:24.486137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08320150-995c-4ae5-b458-239de2ab5b4d admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:24.559470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50caedd-e391-4cec-a6ef-b14935c83086 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:24.559968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50caedd-e391-4cec-a6ef-b14935c83086 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:24.560384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50caedd-e391-4cec-a6ef-b14935c83086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:24.595450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e15d0d-a566-4576-9b8b-797e50682bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:24.597697 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e15d0d-a566-4576-9b8b-797e50682bfd admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:24.603631 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e15d0d-a566-4576-9b8b-797e50682bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:24.604352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e15d0d-a566-4576-9b8b-797e50682bfd admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:24.605309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e15d0d-a566-4576-9b8b-797e50682bfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:24.608342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e15d0d-a566-4576-9b8b-797e50682bfd admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:24.619155 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-e50caedd-e391-4cec-a6ef-b14935c83086 None None] MFA Rules not processed for user `acfba24ac674454d86e415b38c0c4618`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:24.620479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e50caedd-e391-4cec-a6ef-b14935c83086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:24.646756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1090/2158] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:52:24 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:52:24.651932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e15d0d-a566-4576-9b8b-797e50682bfd admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:24.652105 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:24.669141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e15d0d-a566-4576-9b8b-797e50682bfd admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:24.676913 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26149d6-ed28-4481-bbff-891792694367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:24.677873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26149d6-ed28-4481-bbff-891792694367 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:24.678397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26149d6-ed28-4481-bbff-891792694367 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:24.678880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26149d6-ed28-4481-bbff-891792694367 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:24.680445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26149d6-ed28-4481-bbff-891792694367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:24.683843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26149d6-ed28-4481-bbff-891792694367 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:24.701747 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1091/2159] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:24.711909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26149d6-ed28-4481-bbff-891792694367 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:24.728093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1069/2160] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:24.728370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26149d6-ed28-4481-bbff-891792694367 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:24.731180 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1161ef-3d06-4de9-974c-88914e963205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:24.731817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1161ef-3d06-4de9-974c-88914e963205 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:24.732095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1161ef-3d06-4de9-974c-88914e963205 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:24.732391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1161ef-3d06-4de9-974c-88914e963205 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:24.733178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1161ef-3d06-4de9-974c-88914e963205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:24.734821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1161ef-3d06-4de9-974c-88914e963205 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:24.739713 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50caedd-e391-4cec-a6ef-b14935c83086 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:24.741231 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e50caedd-e391-4cec-a6ef-b14935c83086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:24.754308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1092/2161] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:24.758940 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1161ef-3d06-4de9-974c-88914e963205 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:24.770122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1161ef-3d06-4de9-974c-88914e963205 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:24.776123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c00d99-aef3-4401-aeeb-5c93b9f3268f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:24.776810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c00d99-aef3-4401-aeeb-5c93b9f3268f admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:24.777090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c00d99-aef3-4401-aeeb-5c93b9f3268f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:24.777367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c00d99-aef3-4401-aeeb-5c93b9f3268f admin admin] PATH_INFO: `/v3/projects/a1c6c40e62d8456b98ffe4895a881f62/users/175c428343ef409fad1c2fa191faa7d8/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:24.795500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c00d99-aef3-4401-aeeb-5c93b9f3268f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1c6c40e62d8456b98ffe4895a881f62, user_id=175c428343ef409fad1c2fa191faa7d8, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:24.799513 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c00d99-aef3-4401-aeeb-5c93b9f3268f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:24.820019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1093/2162] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:52:24 2022] PUT /identity/v3/projects/a1c6c40e62d8456b98ffe4895a881f62/users/175c428343ef409fad1c2fa191faa7d8/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:52:24.833577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5a872-d46e-485f-a3d8-068993148574 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:24.833577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5a872-d46e-485f-a3d8-068993148574 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:24.833906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5a872-d46e-485f-a3d8-068993148574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:24.840830 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50caedd-e391-4cec-a6ef-b14935c83086 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:24.866377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-cbe5a872-d46e-485f-a3d8-068993148574 None None] MFA Rules not processed for user `175c428343ef409fad1c2fa191faa7d8`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:24.868299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-cbe5a872-d46e-485f-a3d8-068993148574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:24.909102 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76fcafd-0973-48eb-8b79-dd0444cffab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:24.909867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fcafd-0973-48eb-8b79-dd0444cffab7 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:24.910211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fcafd-0973-48eb-8b79-dd0444cffab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:24.910657 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fcafd-0973-48eb-8b79-dd0444cffab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:24.940006 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:24.944525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1094/2163] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:24.950474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5a872-d46e-485f-a3d8-068993148574 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:24.983053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76fcafd-0973-48eb-8b79-dd0444cffab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:24.984487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76fcafd-0973-48eb-8b79-dd0444cffab7 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:24.997226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1070/2164] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:25.009553 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76fcafd-0973-48eb-8b79-dd0444cffab7 service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:25.011698 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b76fcafd-0973-48eb-8b79-dd0444cffab7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:25.019821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5a872-d46e-485f-a3d8-068993148574 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:25.025201 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad165539-d67f-4ef4-8a0c-8c6447982268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:25.026129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad165539-d67f-4ef4-8a0c-8c6447982268 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.026407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad165539-d67f-4ef4-8a0c-8c6447982268 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.026742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad165539-d67f-4ef4-8a0c-8c6447982268 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.028058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad165539-d67f-4ef4-8a0c-8c6447982268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:25.030013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad165539-d67f-4ef4-8a0c-8c6447982268 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:25.031582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76fcafd-0973-48eb-8b79-dd0444cffab7 service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:25.036421 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5254ca5-f58b-4549-9891-b6d42f8d75b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:25.037133 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5254ca5-f58b-4549-9891-b6d42f8d75b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.037484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5254ca5-f58b-4549-9891-b6d42f8d75b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.037817 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5254ca5-f58b-4549-9891-b6d42f8d75b9 admin admin] PATH_INFO: `/v3/users/1a816b1728944b44951306ff3b23abfa` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.039649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5254ca5-f58b-4549-9891-b6d42f8d75b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a816b1728944b44951306ff3b23abfa)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:25.042378 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5254ca5-f58b-4549-9891-b6d42f8d75b9 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:25.064690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1095/2165] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:52:24 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:52:25.068871 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad165539-d67f-4ef4-8a0c-8c6447982268 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:25.084881 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:25.103076 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-e5254ca5-f58b-4549-9891-b6d42f8d75b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a816b1728944b44951306ff3b23abfa'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:25.115555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-e5254ca5-f58b-4549-9891-b6d42f8d75b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a816b1728944b44951306ff3b23abfa'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:25.116570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-e5254ca5-f58b-4549-9891-b6d42f8d75b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a816b1728944b44951306ff3b23abfa'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:25.135475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-e5254ca5-f58b-4549-9891-b6d42f8d75b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a816b1728944b44951306ff3b23abfa'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:25.144412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad165539-d67f-4ef4-8a0c-8c6447982268 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:25.146098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-e5254ca5-f58b-4549-9891-b6d42f8d75b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a816b1728944b44951306ff3b23abfa'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:25.156219 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a349af8-930a-4d36-8fe8-820b4cafb2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:25.156862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a349af8-930a-4d36-8fe8-820b4cafb2d4 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.157134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a349af8-930a-4d36-8fe8-820b4cafb2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.157600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a349af8-930a-4d36-8fe8-820b4cafb2d4 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.158279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a349af8-930a-4d36-8fe8-820b4cafb2d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:25.160489 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a349af8-930a-4d36-8fe8-820b4cafb2d4 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:25.161132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1071/2166] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:52:25 2022] DELETE /identity/v3/users/1a816b1728944b44951306ff3b23abfa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:25.166287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70a6a5-2e70-4f86-a8da-9958ebded345 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.166913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70a6a5-2e70-4f86-a8da-9958ebded345 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.167337 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70a6a5-2e70-4f86-a8da-9958ebded345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.187010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1096/2167] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:52:25 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:52:25.194885 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a349af8-930a-4d36-8fe8-820b4cafb2d4 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:25.197645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-9d70a6a5-2e70-4f86-a8da-9958ebded345 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:25.197959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4a349af8-930a-4d36-8fe8-820b4cafb2d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:25.199314 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9d70a6a5-2e70-4f86-a8da-9958ebded345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:25.211783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a349af8-930a-4d36-8fe8-820b4cafb2d4 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:25.291931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1072/2168] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:25.308001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70a6a5-2e70-4f86-a8da-9958ebded345 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:25.310179 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9d70a6a5-2e70-4f86-a8da-9958ebded345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:25.317055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880f300-667c-4f49-8c70-832b606fae81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:25.319747 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880f300-667c-4f49-8c70-832b606fae81 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.320199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880f300-667c-4f49-8c70-832b606fae81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.321771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880f300-667c-4f49-8c70-832b606fae81 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.323022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f880f300-667c-4f49-8c70-832b606fae81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:25.328362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f880f300-667c-4f49-8c70-832b606fae81 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:25.345839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1097/2169] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:25.347115 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880f300-667c-4f49-8c70-832b606fae81 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:25.361531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880f300-667c-4f49-8c70-832b606fae81 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:25.368171 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f818615-dc13-4407-ab46-1c9915ec4809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:25.369023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f818615-dc13-4407-ab46-1c9915ec4809 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.369307 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f818615-dc13-4407-ab46-1c9915ec4809 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.369687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f818615-dc13-4407-ab46-1c9915ec4809 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.370603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f818615-dc13-4407-ab46-1c9915ec4809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:25.372366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f818615-dc13-4407-ab46-1c9915ec4809 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:25.383981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:25.400339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1098/2170] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:25.401463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f818615-dc13-4407-ab46-1c9915ec4809 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:25.413681 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f818615-dc13-4407-ab46-1c9915ec4809 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:25.419842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462307b-9fd0-4ae8-afaf-2f84c95cfceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:25.420669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462307b-9fd0-4ae8-afaf-2f84c95cfceb admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.420993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462307b-9fd0-4ae8-afaf-2f84c95cfceb admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.421316 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462307b-9fd0-4ae8-afaf-2f84c95cfceb admin admin] PATH_INFO: `/v3/projects/c7f6010bf0cb4b1cbc2dcf8e62310c90/users/bdad79a1b4774695b2d12bd8c23dc216/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.433904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70a6a5-2e70-4f86-a8da-9958ebded345 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:25.444464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462307b-9fd0-4ae8-afaf-2f84c95cfceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7f6010bf0cb4b1cbc2dcf8e62310c90, user_id=bdad79a1b4774695b2d12bd8c23dc216, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:25.447142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520c8b1-415f-4941-bc80-f2f3d712c96a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:25.447966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462307b-9fd0-4ae8-afaf-2f84c95cfceb admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:25.448230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520c8b1-415f-4941-bc80-f2f3d712c96a service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.448758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520c8b1-415f-4941-bc80-f2f3d712c96a service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.449242 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520c8b1-415f-4941-bc80-f2f3d712c96a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.469748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520c8b1-415f-4941-bc80-f2f3d712c96a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:25.472250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520c8b1-415f-4941-bc80-f2f3d712c96a service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:25.474406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1099/2171] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:52:25 2022] PUT /identity/v3/projects/c7f6010bf0cb4b1cbc2dcf8e62310c90/users/bdad79a1b4774695b2d12bd8c23dc216/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:25.493526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2b35f-4e4e-45fb-a151-a0bc631e06b5 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.494210 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2b35f-4e4e-45fb-a151-a0bc631e06b5 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.494490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2b35f-4e4e-45fb-a151-a0bc631e06b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.496902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1073/2172] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:25.532812 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-b0a2b35f-4e4e-45fb-a151-a0bc631e06b5 None None] MFA Rules not processed for user `bdad79a1b4774695b2d12bd8c23dc216`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:25.534134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b0a2b35f-4e4e-45fb-a151-a0bc631e06b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:25.621881 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1100/2173] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:52:25.626823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520c8b1-415f-4941-bc80-f2f3d712c96a service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:25.638822 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520c8b1-415f-4941-bc80-f2f3d712c96a service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:25.699256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:25.712082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d506506-b8c3-4abd-b0a7-e70bbe072f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:25.713184 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d506506-b8c3-4abd-b0a7-e70bbe072f30 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.713505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d506506-b8c3-4abd-b0a7-e70bbe072f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.713752 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d506506-b8c3-4abd-b0a7-e70bbe072f30 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.714653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d506506-b8c3-4abd-b0a7-e70bbe072f30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:25.716982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d506506-b8c3-4abd-b0a7-e70bbe072f30 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:25.752412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1074/2174] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:52:25 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:52:25.756634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2b35f-4e4e-45fb-a151-a0bc631e06b5 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:25.777180 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2b35f-4e4e-45fb-a151-a0bc631e06b5 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:25.866910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76c751-a22e-4758-be7b-bad05fab610f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:25.868089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76c751-a22e-4758-be7b-bad05fab610f admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.868998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76c751-a22e-4758-be7b-bad05fab610f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.869271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76c751-a22e-4758-be7b-bad05fab610f admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.869944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76c751-a22e-4758-be7b-bad05fab610f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:25.871907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76c751-a22e-4758-be7b-bad05fab610f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:25.895788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1101/2175] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:52:25 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:52:25.900378 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d506506-b8c3-4abd-b0a7-e70bbe072f30 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:25.918015 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d506506-b8c3-4abd-b0a7-e70bbe072f30 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:25.918886 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76c751-a22e-4758-be7b-bad05fab610f admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:25.920946 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-db76c751-a22e-4758-be7b-bad05fab610f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:25.928039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba48398-f80d-4e51-b283-1fabe06f941b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:25.928970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba48398-f80d-4e51-b283-1fabe06f941b admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.929253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba48398-f80d-4e51-b283-1fabe06f941b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.929581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba48398-f80d-4e51-b283-1fabe06f941b admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.932126 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba48398-f80d-4e51-b283-1fabe06f941b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:25.933766 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba48398-f80d-4e51-b283-1fabe06f941b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:25.936895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76c751-a22e-4758-be7b-bad05fab610f admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:25.943554 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371577c-a453-42ae-82eb-f9dc5b25548a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:25.944486 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371577c-a453-42ae-82eb-f9dc5b25548a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.944971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371577c-a453-42ae-82eb-f9dc5b25548a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.945134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371577c-a453-42ae-82eb-f9dc5b25548a admin admin] PATH_INFO: `/v3/projects/7e5d947dbe804f0db311a37d9cca6c5a` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.947636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371577c-a453-42ae-82eb-f9dc5b25548a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e5d947dbe804f0db311a37d9cca6c5a)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:25.950663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371577c-a453-42ae-82eb-f9dc5b25548a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:25.950833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1075/2176] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:25.954547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba48398-f80d-4e51-b283-1fabe06f941b admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:25.971487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba48398-f80d-4e51-b283-1fabe06f941b admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:25.977944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe707a2f-647f-4935-ab86-732b4f02d604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:25.978698 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe707a2f-647f-4935-ab86-732b4f02d604 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:25.978979 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe707a2f-647f-4935-ab86-732b4f02d604 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:25.979252 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe707a2f-647f-4935-ab86-732b4f02d604 admin admin] PATH_INFO: `/v3/projects/0817cd7504774a928b4130361977376f/users/0a6f3e28fd414f2d9748aa967f014490/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:25.999078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe707a2f-647f-4935-ab86-732b4f02d604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0817cd7504774a928b4130361977376f, user_id=0a6f3e28fd414f2d9748aa967f014490, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:26.001978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe707a2f-647f-4935-ab86-732b4f02d604 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:26.017370 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b371577c-a453-42ae-82eb-f9dc5b25548a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e5d947dbe804f0db311a37d9cca6c5a', 'request_id': 'req-b371577c-a453-42ae-82eb-f9dc5b25548a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:26.018296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1076/2177] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:52:25 2022] PUT /identity/v3/projects/0817cd7504774a928b4130361977376f/users/0a6f3e28fd414f2d9748aa967f014490/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:26.018949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:26.022135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe707a2f-647f-4935-ab86-732b4f02d604 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:26.031746 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b371577c-a453-42ae-82eb-f9dc5b25548a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e5d947dbe804f0db311a37d9cca6c5a', 'request_id': 'req-b371577c-a453-42ae-82eb-f9dc5b25548a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:26.048134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b371577c-a453-42ae-82eb-f9dc5b25548a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e5d947dbe804f0db311a37d9cca6c5a', 'request_id': 'req-b371577c-a453-42ae-82eb-f9dc5b25548a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:26.066972 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b371577c-a453-42ae-82eb-f9dc5b25548a admin admin] The token cache is being invalidate because project 7e5d947dbe804f0db311a37d9cca6c5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:52:26.067343 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b371577c-a453-42ae-82eb-f9dc5b25548a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:26.067943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b371577c-a453-42ae-82eb-f9dc5b25548a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:26.069998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1102/2178] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:52:25 2022] DELETE /identity/v3/projects/7e5d947dbe804f0db311a37d9cca6c5a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:26.075293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de355b9-b0e5-48f4-9533-a51ff8cf33ae None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.075891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de355b9-b0e5-48f4-9533-a51ff8cf33ae None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.076368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de355b9-b0e5-48f4-9533-a51ff8cf33ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.102948 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe707a2f-647f-4935-ab86-732b4f02d604 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:26.105093 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-2de355b9-b0e5-48f4-9533-a51ff8cf33ae None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:26.106579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2de355b9-b0e5-48f4-9533-a51ff8cf33ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:26.108249 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b294c-106d-424c-b844-ee765b367b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:26.109074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b294c-106d-424c-b844-ee765b367b2a admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.109356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b294c-106d-424c-b844-ee765b367b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.109626 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b294c-106d-424c-b844-ee765b367b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.110493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b294c-106d-424c-b844-ee765b367b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:26.112316 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b294c-106d-424c-b844-ee765b367b2a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:26.126521 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1103/2179] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:52:26.128469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1077/2180] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:26.131093 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366562b9-c6cf-4f05-afa5-a65169d4aa51 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.131093 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366562b9-c6cf-4f05-afa5-a65169d4aa51 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.131093 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366562b9-c6cf-4f05-afa5-a65169d4aa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.136442 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b294c-106d-424c-b844-ee765b367b2a admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:26.138212 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c69b294c-106d-424c-b844-ee765b367b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:26.152668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b294c-106d-424c-b844-ee765b367b2a admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:26.155507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-366562b9-c6cf-4f05-afa5-a65169d4aa51 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:26.157623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-366562b9-c6cf-4f05-afa5-a65169d4aa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:26.158298 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42882cf4-74ac-41c2-bef6-90cc3787c55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:26.159124 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42882cf4-74ac-41c2-bef6-90cc3787c55c admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.159502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42882cf4-74ac-41c2-bef6-90cc3787c55c admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.159779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42882cf4-74ac-41c2-bef6-90cc3787c55c admin admin] PATH_INFO: `/v3/projects/0817cd7504774a928b4130361977376f/users/0a6f3e28fd414f2d9748aa967f014490/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.163808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42882cf4-74ac-41c2-bef6-90cc3787c55c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0817cd7504774a928b4130361977376f, user_id=0a6f3e28fd414f2d9748aa967f014490, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:26.167133 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42882cf4-74ac-41c2-bef6-90cc3787c55c admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:26.171141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1104/2181] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:26.175998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366562b9-c6cf-4f05-afa5-a65169d4aa51 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:26.182965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1078/2182] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:52:26 2022] PUT /identity/v3/projects/0817cd7504774a928b4130361977376f/users/0a6f3e28fd414f2d9748aa967f014490/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:26.191615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366562b9-c6cf-4f05-afa5-a65169d4aa51 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:26.194513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64326910-4cb3-4ac0-aa76-5d8036892aa5 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.194837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64326910-4cb3-4ac0-aa76-5d8036892aa5 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.195165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64326910-4cb3-4ac0-aa76-5d8036892aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.246966 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-64326910-4cb3-4ac0-aa76-5d8036892aa5 None None] MFA Rules not processed for user `0a6f3e28fd414f2d9748aa967f014490`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:26.248917 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-64326910-4cb3-4ac0-aa76-5d8036892aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:26.278407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afe7bf-d7ac-4dc6-bad5-6eb3cdf113bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:26.279334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afe7bf-d7ac-4dc6-bad5-6eb3cdf113bd admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.279778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afe7bf-d7ac-4dc6-bad5-6eb3cdf113bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.280052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afe7bf-d7ac-4dc6-bad5-6eb3cdf113bd admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.280755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afe7bf-d7ac-4dc6-bad5-6eb3cdf113bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:52:26.281059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afe7bf-d7ac-4dc6-bad5-6eb3cdf113bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:26.282680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afe7bf-d7ac-4dc6-bad5-6eb3cdf113bd admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:26.296055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1105/2183] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:52:26.301710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afe7bf-d7ac-4dc6-bad5-6eb3cdf113bd admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:26.317121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afe7bf-d7ac-4dc6-bad5-6eb3cdf113bd admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:26.325181 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74fb2b-8718-44a8-a5bc-9101dec56c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:26.325972 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74fb2b-8718-44a8-a5bc-9101dec56c71 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.326246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74fb2b-8718-44a8-a5bc-9101dec56c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.326512 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74fb2b-8718-44a8-a5bc-9101dec56c71 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.327208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74fb2b-8718-44a8-a5bc-9101dec56c71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:26.329169 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74fb2b-8718-44a8-a5bc-9101dec56c71 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:26.342233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1079/2184] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:26.357738 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64326910-4cb3-4ac0-aa76-5d8036892aa5 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:26.362713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1106/2185] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:52:26 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:52:26.367621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74fb2b-8718-44a8-a5bc-9101dec56c71 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:26.369773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-64326910-4cb3-4ac0-aa76-5d8036892aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:26.383516 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:26.437824 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64326910-4cb3-4ac0-aa76-5d8036892aa5 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:26.438145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74fb2b-8718-44a8-a5bc-9101dec56c71 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:26.443035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c64c25-2361-4997-b2b2-fcdbabe0a8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:26.443586 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c64c25-2361-4997-b2b2-fcdbabe0a8f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.443858 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c64c25-2361-4997-b2b2-fcdbabe0a8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.444147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c64c25-2361-4997-b2b2-fcdbabe0a8f7 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.445063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c64c25-2361-4997-b2b2-fcdbabe0a8f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:26.447353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c64c25-2361-4997-b2b2-fcdbabe0a8f7 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:26.479809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1107/2186] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:52:26 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:52:26.484253 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c64c25-2361-4997-b2b2-fcdbabe0a8f7 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:26.497648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c64c25-2361-4997-b2b2-fcdbabe0a8f7 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:26.503169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f8bd1-2b2e-4db7-9f8c-29a54c2a9b35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:26.503837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f8bd1-2b2e-4db7-9f8c-29a54c2a9b35 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.504142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f8bd1-2b2e-4db7-9f8c-29a54c2a9b35 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.504413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f8bd1-2b2e-4db7-9f8c-29a54c2a9b35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.504647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c193e-e8a6-4327-ac3b-a29976b5dc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:26.505604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c193e-e8a6-4327-ac3b-a29976b5dc42 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.505891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c193e-e8a6-4327-ac3b-a29976b5dc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.506167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c193e-e8a6-4327-ac3b-a29976b5dc42 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.506303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ce5f8bd1-2b2e-4db7-9f8c-29a54c2a9b35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:26.507130 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c193e-e8a6-4327-ac3b-a29976b5dc42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:26.509405 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c193e-e8a6-4327-ac3b-a29976b5dc42 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:26.520683 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1108/2187] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:26.526032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c193e-e8a6-4327-ac3b-a29976b5dc42 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:26.537012 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c193e-e8a6-4327-ac3b-a29976b5dc42 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:26.544374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3c178-74f2-4da9-bc0c-1146bca69266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:26.545525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3c178-74f2-4da9-bc0c-1146bca69266 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.546051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3c178-74f2-4da9-bc0c-1146bca69266 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.546590 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3c178-74f2-4da9-bc0c-1146bca69266 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.547718 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3c178-74f2-4da9-bc0c-1146bca69266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:26.549590 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3c178-74f2-4da9-bc0c-1146bca69266 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:26.569032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1109/2188] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:52:26.570139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3c178-74f2-4da9-bc0c-1146bca69266 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:26.578386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:26.582964 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3c178-74f2-4da9-bc0c-1146bca69266 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:26.583513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f8bd1-2b2e-4db7-9f8c-29a54c2a9b35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:26.585078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f8bd1-2b2e-4db7-9f8c-29a54c2a9b35 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:26.591434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2242f6-96d0-4d29-9d42-7ff9713a6a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:26.592283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2242f6-96d0-4d29-9d42-7ff9713a6a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.593073 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2242f6-96d0-4d29-9d42-7ff9713a6a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.593422 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2242f6-96d0-4d29-9d42-7ff9713a6a41 admin admin] PATH_INFO: `/v3/projects/60b84d954e3c4fc98d014a4cd954084b/users/53296d6ae1594186a435e6ed217e1aff/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.602167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1080/2189] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:26.615758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2242f6-96d0-4d29-9d42-7ff9713a6a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b84d954e3c4fc98d014a4cd954084b, user_id=53296d6ae1594186a435e6ed217e1aff, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:26.619543 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2242f6-96d0-4d29-9d42-7ff9713a6a41 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:26.644289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1110/2190] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:52:26 2022] PUT /identity/v3/projects/60b84d954e3c4fc98d014a4cd954084b/users/53296d6ae1594186a435e6ed217e1aff/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:52:26.660624 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2531e-e7f9-4741-b260-585445fde972 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.661869 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2531e-e7f9-4741-b260-585445fde972 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.662307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2531e-e7f9-4741-b260-585445fde972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.702670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-e5c2531e-e7f9-4741-b260-585445fde972 None None] MFA Rules not processed for user `53296d6ae1594186a435e6ed217e1aff`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:26.703227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e5c2531e-e7f9-4741-b260-585445fde972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:26.753061 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c3f02-a3b3-4883-bf8d-140ed9272504 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:26.753435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c3f02-a3b3-4883-bf8d-140ed9272504 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:26.753789 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c3f02-a3b3-4883-bf8d-140ed9272504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:26.782084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-aa5c3f02-a3b3-4883-bf8d-140ed9272504 None None] MFA Rules not processed for user `acfba24ac674454d86e415b38c0c4618`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:26.783853 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-aa5c3f02-a3b3-4883-bf8d-140ed9272504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:26.815805 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1081/2191] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:26.873699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2531e-e7f9-4741-b260-585445fde972 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:26.879439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e5c2531e-e7f9-4741-b260-585445fde972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:26.886321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1111/2192] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:52:26.902149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:26.979341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2531e-e7f9-4741-b260-585445fde972 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:27.001474 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ecd20-b3e6-47d6-9b8c-f59dc5ed7384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:27.002118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ecd20-b3e6-47d6-9b8c-f59dc5ed7384 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:27.002394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ecd20-b3e6-47d6-9b8c-f59dc5ed7384 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:27.002684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ecd20-b3e6-47d6-9b8c-f59dc5ed7384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:27.013596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ecd20-b3e6-47d6-9b8c-f59dc5ed7384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:27.015304 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ecd20-b3e6-47d6-9b8c-f59dc5ed7384 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:27.027562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1082/2193] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:28.392037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c3f02-a3b3-4883-bf8d-140ed9272504 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:28.405205 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c3f02-a3b3-4883-bf8d-140ed9272504 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:28.411675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8454733-5b20-48a3-a174-042b490c30b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:28.412284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8454733-5b20-48a3-a174-042b490c30b4 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:28.412575 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8454733-5b20-48a3-a174-042b490c30b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:28.412828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8454733-5b20-48a3-a174-042b490c30b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:28.416301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d8454733-5b20-48a3-a174-042b490c30b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:28.487971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8454733-5b20-48a3-a174-042b490c30b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:28.489461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8454733-5b20-48a3-a174-042b490c30b4 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:28.499756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1112/2194] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:29.219889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ecd20-b3e6-47d6-9b8c-f59dc5ed7384 service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:29.233065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ecd20-b3e6-47d6-9b8c-f59dc5ed7384 service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:29.237248 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1c33d-c745-4382-895d-1ff6a2b750cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:29.238048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1c33d-c745-4382-895d-1ff6a2b750cf service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:29.238382 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1c33d-c745-4382-895d-1ff6a2b750cf service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:29.238750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1c33d-c745-4382-895d-1ff6a2b750cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:29.324075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1c33d-c745-4382-895d-1ff6a2b750cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:29.325735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1c33d-c745-4382-895d-1ff6a2b750cf service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:29.337650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1083/2195] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:33.441511 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8454733-5b20-48a3-a174-042b490c30b4 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:33.589818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8454733-5b20-48a3-a174-042b490c30b4 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:33.595041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:33.595755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:33.596035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:33.596305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] PATH_INFO: `/v3/users/175c428343ef409fad1c2fa191faa7d8` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:33.597588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=175c428343ef409fad1c2fa191faa7d8)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:33.600032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:33.647548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:33.687041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '175c428343ef409fad1c2fa191faa7d8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:33.725162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '175c428343ef409fad1c2fa191faa7d8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:33.749448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '175c428343ef409fad1c2fa191faa7d8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:33.750186 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '175c428343ef409fad1c2fa191faa7d8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:33.750921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '175c428343ef409fad1c2fa191faa7d8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:33.776787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1113/2196] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:52:33 2022] DELETE /identity/v3/users/175c428343ef409fad1c2fa191faa7d8 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:33.778614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84365dc1-d946-479b-ac46-044fa34df4a9 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:33.779205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84365dc1-d946-479b-ac46-044fa34df4a9 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:33.779205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84365dc1-d946-479b-ac46-044fa34df4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:33.814430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-84365dc1-d946-479b-ac46-044fa34df4a9 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:33.817402 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-84365dc1-d946-479b-ac46-044fa34df4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:33.902275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1084/2197] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:33.912442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:33.913900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:34.015565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd94ed-64a3-466e-99d1-e583224817eb admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:34.020276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970e531-8780-403f-990a-1f1ffa3e2fe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:34.020782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970e531-8780-403f-990a-1f1ffa3e2fe1 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:34.021063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970e531-8780-403f-990a-1f1ffa3e2fe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:34.021334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970e531-8780-403f-990a-1f1ffa3e2fe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:34.032787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970e531-8780-403f-990a-1f1ffa3e2fe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:34.034421 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970e531-8780-403f-990a-1f1ffa3e2fe1 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:34.048336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1114/2198] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:34.396028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84365dc1-d946-479b-ac46-044fa34df4a9 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:34.397829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-84365dc1-d946-479b-ac46-044fa34df4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:34.411712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84365dc1-d946-479b-ac46-044fa34df4a9 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:34.433413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:34.436108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:34.436331 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:34.442880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] PATH_INFO: `/v3/projects/a1c6c40e62d8456b98ffe4895a881f62` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:34.444967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1c6c40e62d8456b98ffe4895a881f62)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:34.447862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:34.555282 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:34.565379 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1c6c40e62d8456b98ffe4895a881f62', 'request_id': 'req-441498f2-1c89-4032-95f8-0e05428c2190'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:34.594359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1c6c40e62d8456b98ffe4895a881f62', 'request_id': 'req-441498f2-1c89-4032-95f8-0e05428c2190'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:34.612185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1c6c40e62d8456b98ffe4895a881f62', 'request_id': 'req-441498f2-1c89-4032-95f8-0e05428c2190'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:34.627530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] The token cache is being invalidate because project a1c6c40e62d8456b98ffe4895a881f62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:52:34.627954 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:34.628900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:34.635803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1085/2199] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:52:34 2022] DELETE /identity/v3/projects/a1c6c40e62d8456b98ffe4895a881f62 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:34.637006 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970e531-8780-403f-990a-1f1ffa3e2fe1 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:34.653195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970e531-8780-403f-990a-1f1ffa3e2fe1 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:34.789208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:34.789208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:34.789208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:34.790167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] PATH_INFO: `/v3/users/bdad79a1b4774695b2d12bd8c23dc216` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:34.792916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdad79a1b4774695b2d12bd8c23dc216)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:34.796288 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:34.912137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdad79a1b4774695b2d12bd8c23dc216'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:34.913714 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:34.941749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdad79a1b4774695b2d12bd8c23dc216'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:34.960780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdad79a1b4774695b2d12bd8c23dc216'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:34.964062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdad79a1b4774695b2d12bd8c23dc216'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:34.964708 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdad79a1b4774695b2d12bd8c23dc216'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:34.982315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1115/2200] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:52:34 2022] DELETE /identity/v3/users/bdad79a1b4774695b2d12bd8c23dc216 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:35.576984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:35.579619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:35.606041 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441498f2-1c89-4032-95f8-0e05428c2190 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:35.673913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df346af0-220e-4210-a9ef-71e816d337e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:35.674557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df346af0-220e-4210-a9ef-71e816d337e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:35.674872 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df346af0-220e-4210-a9ef-71e816d337e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:35.675142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df346af0-220e-4210-a9ef-71e816d337e5 admin admin] PATH_INFO: `/v3/projects/c7f6010bf0cb4b1cbc2dcf8e62310c90` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:35.676334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df346af0-220e-4210-a9ef-71e816d337e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7f6010bf0cb4b1cbc2dcf8e62310c90)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:35.678235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df346af0-220e-4210-a9ef-71e816d337e5 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:35.759001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-df346af0-220e-4210-a9ef-71e816d337e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7f6010bf0cb4b1cbc2dcf8e62310c90', 'request_id': 'req-df346af0-220e-4210-a9ef-71e816d337e5'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:35.770059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-df346af0-220e-4210-a9ef-71e816d337e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7f6010bf0cb4b1cbc2dcf8e62310c90', 'request_id': 'req-df346af0-220e-4210-a9ef-71e816d337e5'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:35.792350 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-df346af0-220e-4210-a9ef-71e816d337e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7f6010bf0cb4b1cbc2dcf8e62310c90', 'request_id': 'req-df346af0-220e-4210-a9ef-71e816d337e5'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:35.803717 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-df346af0-220e-4210-a9ef-71e816d337e5 admin admin] The token cache is being invalidate because project c7f6010bf0cb4b1cbc2dcf8e62310c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:52:35.804090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-df346af0-220e-4210-a9ef-71e816d337e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:35.804820 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-df346af0-220e-4210-a9ef-71e816d337e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:35.807591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1086/2201] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:52:35 2022] DELETE /identity/v3/projects/c7f6010bf0cb4b1cbc2dcf8e62310c90 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:35.811596 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:35.813232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:35.827870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e277d-169c-4517-a037-ebcb046d7fac admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:35.917175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:35.917844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:35.918122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:35.918398 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] PATH_INFO: `/v3/users/0a6f3e28fd414f2d9748aa967f014490` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:35.920205 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a6f3e28fd414f2d9748aa967f014490)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:35.925147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:36.016342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a6f3e28fd414f2d9748aa967f014490'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:36.037393 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a6f3e28fd414f2d9748aa967f014490'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:36.047805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a6f3e28fd414f2d9748aa967f014490'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:36.048550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a6f3e28fd414f2d9748aa967f014490'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:36.049212 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a6f3e28fd414f2d9748aa967f014490'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:36.075114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1116/2202] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:52:35 2022] DELETE /identity/v3/users/0a6f3e28fd414f2d9748aa967f014490 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:36.623114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e2d31-6112-4886-b797-6d487b4ba149 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:36.623666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e2d31-6112-4886-b797-6d487b4ba149 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:36.623666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e2d31-6112-4886-b797-6d487b4ba149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:36.646850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-2e3e2d31-6112-4886-b797-6d487b4ba149 None None] MFA Rules not processed for user `a357a3d27e4e41adae1c56ee39ae5aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:36.648431 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2e3e2d31-6112-4886-b797-6d487b4ba149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:36.710284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1087/2203] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:36.842673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:36.845142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:36.911308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:36.939192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9f890-6fb0-4c8a-b676-a6723f7e68ac admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:36.944634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9721af-42c4-4e70-99b5-a86047933e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:36.945371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9721af-42c4-4e70-99b5-a86047933e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:36.945793 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9721af-42c4-4e70-99b5-a86047933e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:36.946266 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9721af-42c4-4e70-99b5-a86047933e53 admin admin] PATH_INFO: `/v3/projects/0817cd7504774a928b4130361977376f` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:36.947600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9721af-42c4-4e70-99b5-a86047933e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0817cd7504774a928b4130361977376f)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:36.949520 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9721af-42c4-4e70-99b5-a86047933e53 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:37.013479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-be9721af-42c4-4e70-99b5-a86047933e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0817cd7504774a928b4130361977376f', 'request_id': 'req-be9721af-42c4-4e70-99b5-a86047933e53'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:37.021900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-be9721af-42c4-4e70-99b5-a86047933e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0817cd7504774a928b4130361977376f', 'request_id': 'req-be9721af-42c4-4e70-99b5-a86047933e53'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:37.037900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-be9721af-42c4-4e70-99b5-a86047933e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0817cd7504774a928b4130361977376f', 'request_id': 'req-be9721af-42c4-4e70-99b5-a86047933e53'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:37.052228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-be9721af-42c4-4e70-99b5-a86047933e53 admin admin] The token cache is being invalidate because project 0817cd7504774a928b4130361977376f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:52:37.052487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-be9721af-42c4-4e70-99b5-a86047933e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:37.053158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-be9721af-42c4-4e70-99b5-a86047933e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:37.055362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1117/2204] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:52:36 2022] DELETE /identity/v3/projects/0817cd7504774a928b4130361977376f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:37.061367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23f803-ba64-4108-b17d-d51ca4cc42f4 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:37.061736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23f803-ba64-4108-b17d-d51ca4cc42f4 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:37.062190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23f803-ba64-4108-b17d-d51ca4cc42f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:37.078006 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:37.084617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-0d23f803-ba64-4108-b17d-d51ca4cc42f4 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:37.087551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-0d23f803-ba64-4108-b17d-d51ca4cc42f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:37.174750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1088/2205] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:37.235420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de19a22-6d40-4b03-a428-9bf86710efab None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:37.236486 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de19a22-6d40-4b03-a428-9bf86710efab None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:37.236604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de19a22-6d40-4b03-a428-9bf86710efab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:37.260257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-8de19a22-6d40-4b03-a428-9bf86710efab None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:37.261512 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8de19a22-6d40-4b03-a428-9bf86710efab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:37.270128 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1118/2206] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:37.276167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23f803-ba64-4108-b17d-d51ca4cc42f4 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:37.287590 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23f803-ba64-4108-b17d-d51ca4cc42f4 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:37.294355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2ba65-6d3d-4fce-9756-f524d558692e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:37.294355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ba65-6d3d-4fce-9756-f524d558692e admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:37.294355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ba65-6d3d-4fce-9756-f524d558692e admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:37.294948 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2ba65-6d3d-4fce-9756-f524d558692e admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:37.295458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2ba65-6d3d-4fce-9756-f524d558692e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:52:37.295755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2ba65-6d3d-4fce-9756-f524d558692e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:37.297572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2ba65-6d3d-4fce-9756-f524d558692e admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:37.312169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1089/2207] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:52:37.316616 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de19a22-6d40-4b03-a428-9bf86710efab None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:37.327378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de19a22-6d40-4b03-a428-9bf86710efab None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:37.333493 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5181c2-72ec-40cd-a5af-91d42c107b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:37.334186 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5181c2-72ec-40cd-a5af-91d42c107b76 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:37.334444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5181c2-72ec-40cd-a5af-91d42c107b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:37.334838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5181c2-72ec-40cd-a5af-91d42c107b76 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:37.335431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5181c2-72ec-40cd-a5af-91d42c107b76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:37.337829 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5181c2-72ec-40cd-a5af-91d42c107b76 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:37.375494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1119/2208] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:52:37 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:52:37.376855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2ba65-6d3d-4fce-9756-f524d558692e admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:37.389425 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2ba65-6d3d-4fce-9756-f524d558692e admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:37.468126 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b12a86-587c-47fb-b9f8-ef7347f34261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:37.468634 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b12a86-587c-47fb-b9f8-ef7347f34261 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:37.468982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b12a86-587c-47fb-b9f8-ef7347f34261 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:37.469773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b12a86-587c-47fb-b9f8-ef7347f34261 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:37.470353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b12a86-587c-47fb-b9f8-ef7347f34261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:37.472572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b12a86-587c-47fb-b9f8-ef7347f34261 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:37.493958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddf495-a2d2-4c0b-88cd-be16966266a4 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:37.494262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddf495-a2d2-4c0b-88cd-be16966266a4 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:37.494536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddf495-a2d2-4c0b-88cd-be16966266a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:37.507219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1090/2209] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:52:37 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:52:37.512007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b12a86-587c-47fb-b9f8-ef7347f34261 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:37.527322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b12a86-587c-47fb-b9f8-ef7347f34261 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:37.529379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-65ddf495-a2d2-4c0b-88cd-be16966266a4 None None] MFA Rules not processed for user `d183fcdae86448caa81c09823dd3884e`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:37.530586 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-65ddf495-a2d2-4c0b-88cd-be16966266a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:37.533872 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d262423-996b-47ab-bc9f-c144b22c038b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:37.535192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d262423-996b-47ab-bc9f-c144b22c038b admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:37.535731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d262423-996b-47ab-bc9f-c144b22c038b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:37.536241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d262423-996b-47ab-bc9f-c144b22c038b admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:37.537980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d262423-996b-47ab-bc9f-c144b22c038b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:37.542610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d262423-996b-47ab-bc9f-c144b22c038b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:37.558809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1091/2210] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:52:37.562664 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d262423-996b-47ab-bc9f-c144b22c038b admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:37.583276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d262423-996b-47ab-bc9f-c144b22c038b admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:37.591612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed550c-7431-4051-a4f4-2dcff2bd0dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:37.592591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed550c-7431-4051-a4f4-2dcff2bd0dcf admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:37.593122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed550c-7431-4051-a4f4-2dcff2bd0dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:37.593602 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed550c-7431-4051-a4f4-2dcff2bd0dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:37.595519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed550c-7431-4051-a4f4-2dcff2bd0dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:37.599274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed550c-7431-4051-a4f4-2dcff2bd0dcf admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:37.620728 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1092/2211] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:52:37.620728 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed550c-7431-4051-a4f4-2dcff2bd0dcf admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:37.627489 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed550c-7431-4051-a4f4-2dcff2bd0dcf admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:37.636106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1120/2212] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:37.637998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f3ebe-b3e0-465a-97ac-39566de0e291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:37.638787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f3ebe-b3e0-465a-97ac-39566de0e291 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:37.639016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f3ebe-b3e0-465a-97ac-39566de0e291 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:37.639293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f3ebe-b3e0-465a-97ac-39566de0e291 admin admin] PATH_INFO: `/v3/projects/f5e7d85bb21548a68c3e3eae36025f8f/users/0913fdb8852e413e8230882de40c030b/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:37.657870 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f3ebe-b3e0-465a-97ac-39566de0e291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e7d85bb21548a68c3e3eae36025f8f, user_id=0913fdb8852e413e8230882de40c030b, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:37.660564 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f3ebe-b3e0-465a-97ac-39566de0e291 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:37.681831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1093/2213] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:52:37 2022] PUT /identity/v3/projects/f5e7d85bb21548a68c3e3eae36025f8f/users/0913fdb8852e413e8230882de40c030b/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:37.692436 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b0349-433e-4210-ac52-f807e412fb9a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:37.692786 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b0349-433e-4210-ac52-f807e412fb9a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:37.694134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b0349-433e-4210-ac52-f807e412fb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:37.736758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-1f9b0349-433e-4210-ac52-f807e412fb9a None None] MFA Rules not processed for user `0913fdb8852e413e8230882de40c030b`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:37.737998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1f9b0349-433e-4210-ac52-f807e412fb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:37.847258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:37.892384 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1121/2214] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:37.908865 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f3ebe-b3e0-465a-97ac-39566de0e291 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:37.912004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3e8f3ebe-b3e0-465a-97ac-39566de0e291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:37.995974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:38.023475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f3ebe-b3e0-465a-97ac-39566de0e291 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:38.028059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abd8bf-2775-46ae-956d-8e4af7031499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:38.029409 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd8bf-2775-46ae-956d-8e4af7031499 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:38.029885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd8bf-2775-46ae-956d-8e4af7031499 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:38.030355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abd8bf-2775-46ae-956d-8e4af7031499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:38.041999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54abd8bf-2775-46ae-956d-8e4af7031499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:38.043161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54abd8bf-2775-46ae-956d-8e4af7031499 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:38.053175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1094/2215] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:42.832111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504d682-5102-4e05-a30d-c637db722707 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:42.832854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504d682-5102-4e05-a30d-c637db722707 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:42.833230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504d682-5102-4e05-a30d-c637db722707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:42.871839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-9504d682-5102-4e05-a30d-c637db722707 None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:42.874952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9504d682-5102-4e05-a30d-c637db722707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:43.013251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1122/2216] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:52:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:43.098387 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abd8bf-2775-46ae-956d-8e4af7031499 service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:43.100604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-54abd8bf-2775-46ae-956d-8e4af7031499 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:43.378445 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abd8bf-2775-46ae-956d-8e4af7031499 service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:43.398209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7df9c-8e65-4108-a296-4ddf1e8f1be1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:43.399508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7df9c-8e65-4108-a296-4ddf1e8f1be1 service swift] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:43.400796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7df9c-8e65-4108-a296-4ddf1e8f1be1 service swift] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:43.401147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7df9c-8e65-4108-a296-4ddf1e8f1be1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:43.428678 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7df9c-8e65-4108-a296-4ddf1e8f1be1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:43.436476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7df9c-8e65-4108-a296-4ddf1e8f1be1 service swift] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:43.464819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1095/2217] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:52:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:52:43.957339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504d682-5102-4e05-a30d-c637db722707 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:43.960404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9504d682-5102-4e05-a30d-c637db722707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:44.162878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504d682-5102-4e05-a30d-c637db722707 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:44.172720 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f70802-1088-4dca-8720-8cbdf57042c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:44.174916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f70802-1088-4dca-8720-8cbdf57042c1 service placement] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:44.175502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f70802-1088-4dca-8720-8cbdf57042c1 service placement] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:44.175994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f70802-1088-4dca-8720-8cbdf57042c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:44.178528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-56f70802-1088-4dca-8720-8cbdf57042c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:44.215610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f70802-1088-4dca-8720-8cbdf57042c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:44.221028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f70802-1088-4dca-8720-8cbdf57042c1 service placement] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:44.255384 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1123/2218] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:51.537628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7df9c-8e65-4108-a296-4ddf1e8f1be1 service swift] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:51.629329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7df9c-8e65-4108-a296-4ddf1e8f1be1 service swift] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:51.634914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:51.635577 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:51.635847 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:51.636108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] PATH_INFO: `/v3/users/0913fdb8852e413e8230882de40c030b` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:51.637307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0913fdb8852e413e8230882de40c030b)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:51.639337 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:51.706984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0913fdb8852e413e8230882de40c030b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:51.716902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0913fdb8852e413e8230882de40c030b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:51.717666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0913fdb8852e413e8230882de40c030b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:51.742837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:51.746235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0913fdb8852e413e8230882de40c030b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:51.762965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0913fdb8852e413e8230882de40c030b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:51.775790 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1096/2219] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:52:51 2022] DELETE /identity/v3/users/0913fdb8852e413e8230882de40c030b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:52:51.777607 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fb81c-9abe-45ec-8aa2-e7087cd0cc33 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:51.778126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fb81c-9abe-45ec-8aa2-e7087cd0cc33 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:51.778282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fb81c-9abe-45ec-8aa2-e7087cd0cc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:51.805778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-1e8fb81c-9abe-45ec-8aa2-e7087cd0cc33 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:52:51.807326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1e8fb81c-9abe-45ec-8aa2-e7087cd0cc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:51.908456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1124/2220] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:52:51.921152 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:51.922990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:52.011901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8360b-8946-4f41-8bd0-b33794c0bb20 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:52.016389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8498d7-c2f8-4933-bd63-7648078ed186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:52.017270 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8498d7-c2f8-4933-bd63-7648078ed186 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:52.017548 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8498d7-c2f8-4933-bd63-7648078ed186 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:52.017823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8498d7-c2f8-4933-bd63-7648078ed186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:52.033089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8498d7-c2f8-4933-bd63-7648078ed186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:52.034807 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8498d7-c2f8-4933-bd63-7648078ed186 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:52.046422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1097/2221] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:52:52.752963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fb81c-9abe-45ec-8aa2-e7087cd0cc33 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:52:52.754442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1e8fb81c-9abe-45ec-8aa2-e7087cd0cc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:52:52.799773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fb81c-9abe-45ec-8aa2-e7087cd0cc33 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:52:52.817208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:52:52.825029 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:52:52.825029 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:52:52.825029 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59 admin admin] PATH_INFO: `/v3/projects/f5e7d85bb21548a68c3e3eae36025f8f` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:52:52.825029 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e7d85bb21548a68c3e3eae36025f8f)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:52:52.837367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:52:52.870427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:52:52.998300 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e7d85bb21548a68c3e3eae36025f8f', 'request_id': 'req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:53.010334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e7d85bb21548a68c3e3eae36025f8f', 'request_id': 'req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:53.030254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e7d85bb21548a68c3e3eae36025f8f', 'request_id': 'req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:53.046291 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59 admin admin] The token cache is being invalidate because project f5e7d85bb21548a68c3e3eae36025f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:52:53.046618 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:53.047411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-cd39d8cf-0126-4cdc-9d08-8b3e47e93f59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:52:53.057233 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1125/2222] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:52:52 2022] DELETE /identity/v3/projects/f5e7d85bb21548a68c3e3eae36025f8f => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:53:02.041817 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=29,keystone:INSERT=1,keystone:DELETE=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:03.042392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:03.782970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a5587-decd-4d25-85b9-833c2294be17 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:03.783574 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a5587-decd-4d25-85b9-833c2294be17 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:03.783796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a5587-decd-4d25-85b9-833c2294be17 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:03.785557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1098/2223] 10.222.0.86 () {54 vars in 921 bytes} [Fri Aug 5 19:53:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:53:03.792013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755b1bc-d8a6-4eea-95a3-ec51225422b4 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:03.792361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755b1bc-d8a6-4eea-95a3-ec51225422b4 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:03.792645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755b1bc-d8a6-4eea-95a3-ec51225422b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:03.826547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-0755b1bc-d8a6-4eea-95a3-ec51225422b4 None None] MFA Rules not processed for user `ab8f9d9cf4ca4e38ae6eb67e949b40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:03.827810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-0755b1bc-d8a6-4eea-95a3-ec51225422b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:03.908685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1126/2224] 10.222.0.86 () {58 vars in 1018 bytes} [Fri Aug 5 19:53:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:53:03.920918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a5587-decd-4d25-85b9-833c2294be17 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:03.922718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-861a5587-decd-4d25-85b9-833c2294be17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:03.993138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a5587-decd-4d25-85b9-833c2294be17 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:03.998525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad609b-be9f-4a0d-925d-a8edae96035f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72d4459164844e718544d7fde333db76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:03.999172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad609b-be9f-4a0d-925d-a8edae96035f service glance] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:03.999451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad609b-be9f-4a0d-925d-a8edae96035f service glance] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:03.999723 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad609b-be9f-4a0d-925d-a8edae96035f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:04.011967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad609b-be9f-4a0d-925d-a8edae96035f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:04.013440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad609b-be9f-4a0d-925d-a8edae96035f service glance] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:04.026289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1099/2225] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:53:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:53:04.106944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d9608-4d03-41ca-a492-ee0ba87390f4 None None] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:04.107534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d9608-4d03-41ca-a492-ee0ba87390f4 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:04.107534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d9608-4d03-41ca-a492-ee0ba87390f4 None None] PATH_INFO: `/` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:04.108843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1127/2226] 10.222.0.86 () {54 vars in 921 bytes} [Fri Aug 5 19:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:53:04.114043 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c90dca-ea43-4af2-a3ae-efdeb5b5ee6e None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:04.114333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c90dca-ea43-4af2-a3ae-efdeb5b5ee6e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:04.114704 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c90dca-ea43-4af2-a3ae-efdeb5b5ee6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:04.142731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-25c90dca-ea43-4af2-a3ae-efdeb5b5ee6e None None] MFA Rules not processed for user `80097afc3ce540e6a8060f696535e62c`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:04.142731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-25c90dca-ea43-4af2-a3ae-efdeb5b5ee6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:04.200399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1100/2227] 10.222.0.86 () {58 vars in 1018 bytes} [Fri Aug 5 19:53:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:53:04.213885 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d9608-4d03-41ca-a492-ee0ba87390f4 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:04.215696 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b61d9608-4d03-41ca-a492-ee0ba87390f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:04.229869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d9608-4d03-41ca-a492-ee0ba87390f4 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:04.233843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74643f-23ea-4a4e-beb3-81334333a5de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:04.234468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74643f-23ea-4a4e-beb3-81334333a5de service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:04.234973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74643f-23ea-4a4e-beb3-81334333a5de service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:04.235249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74643f-23ea-4a4e-beb3-81334333a5de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:04.244897 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74643f-23ea-4a4e-beb3-81334333a5de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:04.246348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74643f-23ea-4a4e-beb3-81334333a5de service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:04.255276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1128/2228] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:53:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:53:04.350643 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4370d91-7db8-44bd-bd8a-856c4af05b43 None None] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:04.351577 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4370d91-7db8-44bd-bd8a-856c4af05b43 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:04.352079 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4370d91-7db8-44bd-bd8a-856c4af05b43 None None] PATH_INFO: `/` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:04.353470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1101/2229] 10.222.0.86 () {54 vars in 921 bytes} [Fri Aug 5 19:53:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:53:04.359256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842686a-d199-4047-a3bd-afede3b7444c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:04.359619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842686a-d199-4047-a3bd-afede3b7444c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:04.359961 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842686a-d199-4047-a3bd-afede3b7444c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:04.383054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-6842686a-d199-4047-a3bd-afede3b7444c None None] MFA Rules not processed for user `ab8f9d9cf4ca4e38ae6eb67e949b40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:04.384298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6842686a-d199-4047-a3bd-afede3b7444c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:04.395254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1129/2230] 10.222.0.86 () {58 vars in 1018 bytes} [Fri Aug 5 19:53:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:53:04.408618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4370d91-7db8-44bd-bd8a-856c4af05b43 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:04.423125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4370d91-7db8-44bd-bd8a-856c4af05b43 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:04.428982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3597d-71b5-49ae-9a43-251c30028656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:04.429833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3597d-71b5-49ae-9a43-251c30028656 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:04.430119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3597d-71b5-49ae-9a43-251c30028656 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:04.430391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3597d-71b5-49ae-9a43-251c30028656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:04.443929 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3597d-71b5-49ae-9a43-251c30028656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:04.446760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3597d-71b5-49ae-9a43-251c30028656 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:04.460595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1102/2231] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:53:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:53:07.528183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dc9f9-3ea2-4f5f-995a-2e4e57a2f582 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:07.528738 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dc9f9-3ea2-4f5f-995a-2e4e57a2f582 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:07.528738 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dc9f9-3ea2-4f5f-995a-2e4e57a2f582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:07.552533 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-7d7dc9f9-3ea2-4f5f-995a-2e4e57a2f582 None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:07.555335 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7d7dc9f9-3ea2-4f5f-995a-2e4e57a2f582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:07.617561 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1130/2232] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:53:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:53:07.629646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3597d-71b5-49ae-9a43-251c30028656 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:07.631081 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-08c3597d-71b5-49ae-9a43-251c30028656 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:07.706270 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3597d-71b5-49ae-9a43-251c30028656 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:07.712598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996548ef-e387-4c81-ae4d-5fbe3df38897 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:07.713217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996548ef-e387-4c81-ae4d-5fbe3df38897 service swift] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:07.713430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996548ef-e387-4c81-ae4d-5fbe3df38897 service swift] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:07.713694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996548ef-e387-4c81-ae4d-5fbe3df38897 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:07.723774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996548ef-e387-4c81-ae4d-5fbe3df38897 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:07.725378 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996548ef-e387-4c81-ae4d-5fbe3df38897 service swift] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:07.734650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1103/2233] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:53:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:53:09.324874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dc9f9-3ea2-4f5f-995a-2e4e57a2f582 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:09.326394 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7d7dc9f9-3ea2-4f5f-995a-2e4e57a2f582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:09.415238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dc9f9-3ea2-4f5f-995a-2e4e57a2f582 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:09.419982 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:09.420841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:09.421118 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:09.421385 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] PATH_INFO: `/v3/users/53296d6ae1594186a435e6ed217e1aff` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:09.424924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=53296d6ae1594186a435e6ed217e1aff)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:09.426901 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:09.463543 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:09.521976 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53296d6ae1594186a435e6ed217e1aff'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:53:09.547885 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53296d6ae1594186a435e6ed217e1aff'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:53:09.558899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53296d6ae1594186a435e6ed217e1aff'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:53:09.560408 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53296d6ae1594186a435e6ed217e1aff'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:53:09.561060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53296d6ae1594186a435e6ed217e1aff'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:53:09.578038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1131/2234] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:53:09 2022] DELETE /identity/v3/users/53296d6ae1594186a435e6ed217e1aff => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:53:09.583149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16de9e-2223-4e08-8230-7057864bce9c None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:09.584338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16de9e-2223-4e08-8230-7057864bce9c None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:09.594935 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16de9e-2223-4e08-8230-7057864bce9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:09.645233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-db16de9e-2223-4e08-8230-7057864bce9c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:09.647542 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-db16de9e-2223-4e08-8230-7057864bce9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:09.691483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:09.751658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1104/2235] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:53:09.762107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:09.763767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:09.834157 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2f454-395b-49ff-94b3-ae027d8b6e5b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:09.839800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd3ad9-f6d3-4cc8-acdb-4af885a8290b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:09.840661 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd3ad9-f6d3-4cc8-acdb-4af885a8290b service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:09.840945 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd3ad9-f6d3-4cc8-acdb-4af885a8290b service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:09.841771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd3ad9-f6d3-4cc8-acdb-4af885a8290b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:09.853146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd3ad9-f6d3-4cc8-acdb-4af885a8290b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:09.854748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd3ad9-f6d3-4cc8-acdb-4af885a8290b service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:09.867972 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1132/2236] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:53:10.270159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16de9e-2223-4e08-8230-7057864bce9c None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:10.271627 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-db16de9e-2223-4e08-8230-7057864bce9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:10.284943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16de9e-2223-4e08-8230-7057864bce9c None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:10.293258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:10.293915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:10.294211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:10.294560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] PATH_INFO: `/v3/projects/60b84d954e3c4fc98d014a4cd954084b` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:10.295750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b84d954e3c4fc98d014a4cd954084b)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:10.297693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:10.365072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b84d954e3c4fc98d014a4cd954084b', 'request_id': 'req-8a0d2077-c9bd-43e2-930d-c1102df2bd86'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:53:10.371733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b84d954e3c4fc98d014a4cd954084b', 'request_id': 'req-8a0d2077-c9bd-43e2-930d-c1102df2bd86'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:53:10.382861 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b84d954e3c4fc98d014a4cd954084b', 'request_id': 'req-8a0d2077-c9bd-43e2-930d-c1102df2bd86'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:53:10.395328 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] The token cache is being invalidate because project 60b84d954e3c4fc98d014a4cd954084b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:53:10.395614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:53:10.396285 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:53:10.410785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1105/2237] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:53:10 2022] DELETE /identity/v3/projects/60b84d954e3c4fc98d014a4cd954084b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:53:19.861779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=8,keystone:SELECT=46,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:20.392065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:36.772878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07142ac0-413f-46db-98df-de864da2e4ab None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:36.773472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07142ac0-413f-46db-98df-de864da2e4ab None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:36.773579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07142ac0-413f-46db-98df-de864da2e4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:36.796692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-07142ac0-413f-46db-98df-de864da2e4ab None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:53:36.797912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-07142ac0-413f-46db-98df-de864da2e4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:36.855133 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1133/2238] 10.222.0.86 () {58 vars in 967 bytes} [Fri Aug 5 19:53:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:53:36.869015 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:53:36.870581 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:53:36.944963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d2077-c9bd-43e2-930d-c1102df2bd86 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:53:36.953037 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0669c-0a51-4bfd-8ba2-64e0fb83ff0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:53:36.953670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0669c-0a51-4bfd-8ba2-64e0fb83ff0c service swift] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:53:36.953945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0669c-0a51-4bfd-8ba2-64e0fb83ff0c service swift] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:53:36.954737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0669c-0a51-4bfd-8ba2-64e0fb83ff0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:53:36.975949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0669c-0a51-4bfd-8ba2-64e0fb83ff0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:53:36.977398 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0669c-0a51-4bfd-8ba2-64e0fb83ff0c service swift] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:53:36.986477 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1106/2239] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:53:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:53:46.847994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=25 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:53:46.982904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=32 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:01.853929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e63649-33d1-48e1-8e82-e11ece177e05 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:01.854488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e63649-33d1-48e1-8e82-e11ece177e05 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:01.854585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e63649-33d1-48e1-8e82-e11ece177e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:01.875165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-f3e63649-33d1-48e1-8e82-e11ece177e05 None None] MFA Rules not processed for user `acfba24ac674454d86e415b38c0c4618`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:54:01.876376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f3e63649-33d1-48e1-8e82-e11ece177e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:01.935325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1134/2240] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:54:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:54:11.927084 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=25 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:32.796473 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee9075-03c6-48e1-8bff-9d1e3830d31c None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:32.799580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee9075-03c6-48e1-8bff-9d1e3830d31c None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:32.799580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee9075-03c6-48e1-8bff-9d1e3830d31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:32.818445 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-1fee9075-03c6-48e1-8bff-9d1e3830d31c None None] MFA Rules not processed for user `d183fcdae86448caa81c09823dd3884e`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:54:32.819719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-1fee9075-03c6-48e1-8bff-9d1e3830d31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:32.881212 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1107/2241] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:54:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:54:38.443773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e63649-33d1-48e1-8e82-e11ece177e05 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:54:38.445541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f3e63649-33d1-48e1-8e82-e11ece177e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:38.490865 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee9075-03c6-48e1-8bff-9d1e3830d31c None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:54:38.493508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-1fee9075-03c6-48e1-8bff-9d1e3830d31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:38.528887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e63649-33d1-48e1-8e82-e11ece177e05 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:54:38.532840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff981ea-b579-49cc-927b-1373679e5d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:54:38.533549 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff981ea-b579-49cc-927b-1373679e5d33 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:38.534062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff981ea-b579-49cc-927b-1373679e5d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:38.534489 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff981ea-b579-49cc-927b-1373679e5d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:38.540629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1ff981ea-b579-49cc-927b-1373679e5d33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:38.567557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee9075-03c6-48e1-8bff-9d1e3830d31c None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:54:38.573777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46730bd1-412c-484a-ae69-b72440df52f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:54:38.574695 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46730bd1-412c-484a-ae69-b72440df52f8 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:38.575140 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46730bd1-412c-484a-ae69-b72440df52f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:38.575384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46730bd1-412c-484a-ae69-b72440df52f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:38.577540 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-46730bd1-412c-484a-ae69-b72440df52f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:38.642818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff981ea-b579-49cc-927b-1373679e5d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:54:38.645359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff981ea-b579-49cc-927b-1373679e5d33 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:54:38.658195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1135/2242] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:54:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:54:38.681832 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46730bd1-412c-484a-ae69-b72440df52f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:54:38.683320 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46730bd1-412c-484a-ae69-b72440df52f8 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:54:38.697178 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1108/2243] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:54:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:54:38.796362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07ff7f-cc10-49a1-98ad-df665a09be8f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:38.796627 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07ff7f-cc10-49a1-98ad-df665a09be8f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:38.796915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07ff7f-cc10-49a1-98ad-df665a09be8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:38.822726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-4b07ff7f-cc10-49a1-98ad-df665a09be8f None None] MFA Rules not processed for user `a357a3d27e4e41adae1c56ee39ae5aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:54:38.824231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4b07ff7f-cc10-49a1-98ad-df665a09be8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:38.886726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1136/2244] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:54:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:54:45.703616 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9de4b-3400-4e69-b378-014a2d786f7d None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:45.704158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9de4b-3400-4e69-b378-014a2d786f7d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:45.704158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9de4b-3400-4e69-b378-014a2d786f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:45.724497 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-e0a9de4b-3400-4e69-b378-014a2d786f7d None None] MFA Rules not processed for user `e8f7435928a74258af5478959bd939bb`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:54:45.725732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e0a9de4b-3400-4e69-b378-014a2d786f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:45.770173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:45.784677 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1109/2245] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:54:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:54:48.878693 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=79 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:54:53.372435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07ff7f-cc10-49a1-98ad-df665a09be8f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:54:53.373993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4b07ff7f-cc10-49a1-98ad-df665a09be8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:53.391530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07ff7f-cc10-49a1-98ad-df665a09be8f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:54:53.397147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927e8b7-cae0-4679-98d1-e377b75372c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:54:53.397775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927e8b7-cae0-4679-98d1-e377b75372c2 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:54:53.398092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927e8b7-cae0-4679-98d1-e377b75372c2 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:54:53.398898 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927e8b7-cae0-4679-98d1-e377b75372c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:54:53.400714 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c927e8b7-cae0-4679-98d1-e377b75372c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:54:53.412417 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927e8b7-cae0-4679-98d1-e377b75372c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:54:53.414150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927e8b7-cae0-4679-98d1-e377b75372c2 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:54:53.424249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1137/2246] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:54:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:54:55.777763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=4 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:03.419615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=6 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:07.258726 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9de4b-3400-4e69-b378-014a2d786f7d None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:07.260305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e0a9de4b-3400-4e69-b378-014a2d786f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:07.285185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927e8b7-cae0-4679-98d1-e377b75372c2 service nova] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:07.289003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c927e8b7-cae0-4679-98d1-e377b75372c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:07.361511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9de4b-3400-4e69-b378-014a2d786f7d None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:07.370524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea96a8-17a8-4ab1-89cd-86491fbc50ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:07.371680 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea96a8-17a8-4ab1-89cd-86491fbc50ef service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:07.372216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea96a8-17a8-4ab1-89cd-86491fbc50ef service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:07.372747 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea96a8-17a8-4ab1-89cd-86491fbc50ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:07.376062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-0bea96a8-17a8-4ab1-89cd-86491fbc50ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:07.431356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927e8b7-cae0-4679-98d1-e377b75372c2 service nova] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:07.440878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df9e44-b918-49f9-845b-4d717364a3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:07.442026 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df9e44-b918-49f9-845b-4d717364a3b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:07.442612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df9e44-b918-49f9-845b-4d717364a3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:07.443184 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df9e44-b918-49f9-845b-4d717364a3b1 admin admin] PATH_INFO: `/v3/users/acfba24ac674454d86e415b38c0c4618` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:07.446704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df9e44-b918-49f9-845b-4d717364a3b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acfba24ac674454d86e415b38c0c4618)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:07.450200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df9e44-b918-49f9-845b-4d717364a3b1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:07.459546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea96a8-17a8-4ab1-89cd-86491fbc50ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:07.461732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea96a8-17a8-4ab1-89cd-86491fbc50ef service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:07.475510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1110/2247] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:55:07.521436 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-37df9e44-b918-49f9-845b-4d717364a3b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acfba24ac674454d86e415b38c0c4618'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:07.546941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-37df9e44-b918-49f9-845b-4d717364a3b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acfba24ac674454d86e415b38c0c4618'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:07.561999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-37df9e44-b918-49f9-845b-4d717364a3b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acfba24ac674454d86e415b38c0c4618'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:07.562971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-37df9e44-b918-49f9-845b-4d717364a3b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acfba24ac674454d86e415b38c0c4618'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:07.564025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-37df9e44-b918-49f9-845b-4d717364a3b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acfba24ac674454d86e415b38c0c4618'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:07.583772 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1138/2248] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:55:07 2022] DELETE /identity/v3/users/acfba24ac674454d86e415b38c0c4618 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:55:07.843381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea96a8-17a8-4ab1-89cd-86491fbc50ef service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:07.864032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea96a8-17a8-4ab1-89cd-86491fbc50ef service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:07.946762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daaa23-10e0-4d3c-94e9-2c795a669e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:07.946762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daaa23-10e0-4d3c-94e9-2c795a669e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:07.947133 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daaa23-10e0-4d3c-94e9-2c795a669e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:07.947305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daaa23-10e0-4d3c-94e9-2c795a669e53 admin admin] PATH_INFO: `/v3/projects/018e12e98f1143d6acaf348cfea45763` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:07.948765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daaa23-10e0-4d3c-94e9-2c795a669e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=018e12e98f1143d6acaf348cfea45763)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:07.950838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daaa23-10e0-4d3c-94e9-2c795a669e53 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:08.024418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f5daaa23-10e0-4d3c-94e9-2c795a669e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '018e12e98f1143d6acaf348cfea45763', 'request_id': 'req-f5daaa23-10e0-4d3c-94e9-2c795a669e53'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:08.033776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f5daaa23-10e0-4d3c-94e9-2c795a669e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '018e12e98f1143d6acaf348cfea45763', 'request_id': 'req-f5daaa23-10e0-4d3c-94e9-2c795a669e53'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:08.037918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:08.046356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f5daaa23-10e0-4d3c-94e9-2c795a669e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '018e12e98f1143d6acaf348cfea45763', 'request_id': 'req-f5daaa23-10e0-4d3c-94e9-2c795a669e53'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:08.067217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f5daaa23-10e0-4d3c-94e9-2c795a669e53 admin admin] The token cache is being invalidate because project 018e12e98f1143d6acaf348cfea45763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:55:08.067476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f5daaa23-10e0-4d3c-94e9-2c795a669e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:08.068065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f5daaa23-10e0-4d3c-94e9-2c795a669e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:08.071448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1111/2249] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:55:07 2022] DELETE /identity/v3/projects/018e12e98f1143d6acaf348cfea45763 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:55:08.077032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41bf30-3779-414a-9038-16810b93deac None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:08.077401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41bf30-3779-414a-9038-16810b93deac None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:08.077569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41bf30-3779-414a-9038-16810b93deac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:08.106337 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-bc41bf30-3779-414a-9038-16810b93deac None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:08.107619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-bc41bf30-3779-414a-9038-16810b93deac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:08.184226 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1139/2250] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:08.193815 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303403a2-3b6d-4607-8871-13b9128b4eea None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:08.194205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303403a2-3b6d-4607-8871-13b9128b4eea None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:08.194463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303403a2-3b6d-4607-8871-13b9128b4eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:08.217456 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-303403a2-3b6d-4607-8871-13b9128b4eea None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:08.218724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-303403a2-3b6d-4607-8871-13b9128b4eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:08.227740 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1112/2251] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:08.232276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41bf30-3779-414a-9038-16810b93deac None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:08.243302 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41bf30-3779-414a-9038-16810b93deac None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:08.247999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ffbe7-516a-420a-b284-c194a7794711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:08.248631 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ffbe7-516a-420a-b284-c194a7794711 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:08.248906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ffbe7-516a-420a-b284-c194a7794711 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:08.249174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ffbe7-516a-420a-b284-c194a7794711 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:08.249877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ffbe7-516a-420a-b284-c194a7794711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:55:08.250175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ffbe7-516a-420a-b284-c194a7794711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:08.251831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ffbe7-516a-420a-b284-c194a7794711 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:08.264371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1140/2252] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:55:08.268826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303403a2-3b6d-4607-8871-13b9128b4eea None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:08.280389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303403a2-3b6d-4607-8871-13b9128b4eea None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:08.286791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6361b-8138-4919-ad67-67b48765eed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:08.287604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6361b-8138-4919-ad67-67b48765eed0 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:08.287884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6361b-8138-4919-ad67-67b48765eed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:08.288911 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6361b-8138-4919-ad67-67b48765eed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:08.289390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6361b-8138-4919-ad67-67b48765eed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:08.291347 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6361b-8138-4919-ad67-67b48765eed0 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:08.326645 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1113/2253] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:55:08 2022] POST /identity/v3/projects => generated 399 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:55:08.330903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ffbe7-516a-420a-b284-c194a7794711 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:08.344189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ffbe7-516a-420a-b284-c194a7794711 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:08.348949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:08.414607 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd840823-0ba3-4058-bbc7-97747dd7b888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:08.415286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd840823-0ba3-4058-bbc7-97747dd7b888 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:08.415559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd840823-0ba3-4058-bbc7-97747dd7b888 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:08.415835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd840823-0ba3-4058-bbc7-97747dd7b888 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:08.416518 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd840823-0ba3-4058-bbc7-97747dd7b888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:08.418577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd840823-0ba3-4058-bbc7-97747dd7b888 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:08.443835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1141/2254] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:55:08 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:55:08.448860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6361b-8138-4919-ad67-67b48765eed0 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:08.462604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6361b-8138-4919-ad67-67b48765eed0 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:08.467457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c893a-27f9-4fb6-abe9-53f757edd753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:08.468089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c893a-27f9-4fb6-abe9-53f757edd753 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:08.468354 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c893a-27f9-4fb6-abe9-53f757edd753 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:08.468622 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c893a-27f9-4fb6-abe9-53f757edd753 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:08.469313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c893a-27f9-4fb6-abe9-53f757edd753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:08.470969 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c893a-27f9-4fb6-abe9-53f757edd753 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:08.483632 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1114/2255] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:55:08.487821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd840823-0ba3-4058-bbc7-97747dd7b888 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:08.499393 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd840823-0ba3-4058-bbc7-97747dd7b888 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:08.504215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6876b-10b8-49bd-802d-87ec640a910d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:08.504998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6876b-10b8-49bd-802d-87ec640a910d admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:08.505111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6876b-10b8-49bd-802d-87ec640a910d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:08.505376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6876b-10b8-49bd-802d-87ec640a910d admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:08.506114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6876b-10b8-49bd-802d-87ec640a910d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:08.509134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6876b-10b8-49bd-802d-87ec640a910d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:08.521694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1142/2256] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:55:08.526075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c893a-27f9-4fb6-abe9-53f757edd753 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:08.539798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c893a-27f9-4fb6-abe9-53f757edd753 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:08.546540 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4eef7b-f333-41b6-8c77-c2155dd139cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:08.547273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4eef7b-f333-41b6-8c77-c2155dd139cf admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:08.547524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4eef7b-f333-41b6-8c77-c2155dd139cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:08.547798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4eef7b-f333-41b6-8c77-c2155dd139cf admin admin] PATH_INFO: `/v3/projects/a5da932c549c40f894932c2be69781aa/users/fae2d9dc0b804a14a05b83df2299f440/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:08.564206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4eef7b-f333-41b6-8c77-c2155dd139cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5da932c549c40f894932c2be69781aa, user_id=fae2d9dc0b804a14a05b83df2299f440, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:08.566927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4eef7b-f333-41b6-8c77-c2155dd139cf admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:08.581745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1115/2257] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:55:08 2022] PUT /identity/v3/projects/a5da932c549c40f894932c2be69781aa/users/fae2d9dc0b804a14a05b83df2299f440/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:55:08.651009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d12d3-7f3d-4d1f-8626-c93850c2f393 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:08.651292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d12d3-7f3d-4d1f-8626-c93850c2f393 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:08.651744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d12d3-7f3d-4d1f-8626-c93850c2f393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:08.682087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-bc8d12d3-7f3d-4d1f-8626-c93850c2f393 None None] MFA Rules not processed for user `fae2d9dc0b804a14a05b83df2299f440`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:08.683344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-bc8d12d3-7f3d-4d1f-8626-c93850c2f393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:08.751583 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1143/2258] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:55:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:08.763521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4eef7b-f333-41b6-8c77-c2155dd139cf admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:08.765316 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-1e4eef7b-f333-41b6-8c77-c2155dd139cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:08.825173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4eef7b-f333-41b6-8c77-c2155dd139cf admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:08.829194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2cd7c-e89e-4543-9f4e-a403d4260838 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:08.830146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2cd7c-e89e-4543-9f4e-a403d4260838 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:08.830385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2cd7c-e89e-4543-9f4e-a403d4260838 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:08.830992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2cd7c-e89e-4543-9f4e-a403d4260838 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:08.840663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2cd7c-e89e-4543-9f4e-a403d4260838 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:08.842015 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2cd7c-e89e-4543-9f4e-a403d4260838 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:08.853580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1116/2259] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:17.081055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d12d3-7f3d-4d1f-8626-c93850c2f393 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:17.151574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d12d3-7f3d-4d1f-8626-c93850c2f393 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:17.155931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c4a74-cd6d-4704-b3d9-fdfdabd4d494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:17.156601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c4a74-cd6d-4704-b3d9-fdfdabd4d494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:17.156877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c4a74-cd6d-4704-b3d9-fdfdabd4d494 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:17.157203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c4a74-cd6d-4704-b3d9-fdfdabd4d494 admin admin] PATH_INFO: `/v3/users/fae2d9dc0b804a14a05b83df2299f440` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:17.158973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c4a74-cd6d-4704-b3d9-fdfdabd4d494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fae2d9dc0b804a14a05b83df2299f440)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:17.161049 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c4a74-cd6d-4704-b3d9-fdfdabd4d494 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:17.183521 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:17.233167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-862c4a74-cd6d-4704-b3d9-fdfdabd4d494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fae2d9dc0b804a14a05b83df2299f440'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:17.249225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-862c4a74-cd6d-4704-b3d9-fdfdabd4d494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fae2d9dc0b804a14a05b83df2299f440'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:17.265272 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e0e6a-6bef-4f48-9602-de0598204e0c None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:17.265728 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e0e6a-6bef-4f48-9602-de0598204e0c None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:17.265876 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e0e6a-6bef-4f48-9602-de0598204e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:17.269195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-862c4a74-cd6d-4704-b3d9-fdfdabd4d494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fae2d9dc0b804a14a05b83df2299f440'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:17.269827 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-862c4a74-cd6d-4704-b3d9-fdfdabd4d494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fae2d9dc0b804a14a05b83df2299f440'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:17.270493 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-862c4a74-cd6d-4704-b3d9-fdfdabd4d494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fae2d9dc0b804a14a05b83df2299f440'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:17.285678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1144/2260] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:55:17 2022] DELETE /identity/v3/users/fae2d9dc0b804a14a05b83df2299f440 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:55:17.286217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-5d5e0e6a-6bef-4f48-9602-de0598204e0c None None] MFA Rules not processed for user `074f3ddb689c4a60a05d3267080d7e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:17.287599 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5d5e0e6a-6bef-4f48-9602-de0598204e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:17.290860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f66dd-7641-45a5-b8b8-2fe142a2dfbe None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:17.291172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f66dd-7641-45a5-b8b8-2fe142a2dfbe None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:17.291427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f66dd-7641-45a5-b8b8-2fe142a2dfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:17.313646 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e28f66dd-7641-45a5-b8b8-2fe142a2dfbe None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:17.314830 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e28f66dd-7641-45a5-b8b8-2fe142a2dfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:17.368700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1117/2261] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:55:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:55:17.382924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1145/2262] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:55:17.391134 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e0e6a-6bef-4f48-9602-de0598204e0c None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:17.392487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5d5e0e6a-6bef-4f48-9602-de0598204e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:17.428434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:17.459827 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e0e6a-6bef-4f48-9602-de0598204e0c None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:17.464777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e4f54-df61-4c58-b2fb-68a0c21bced9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:17.465435 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e4f54-df61-4c58-b2fb-68a0c21bced9 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:17.465713 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e4f54-df61-4c58-b2fb-68a0c21bced9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:17.465991 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e4f54-df61-4c58-b2fb-68a0c21bced9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:17.479300 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e4f54-df61-4c58-b2fb-68a0c21bced9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:17.481657 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e4f54-df61-4c58-b2fb-68a0c21bced9 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:17.498396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1118/2263] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:17.797227 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f66dd-7641-45a5-b8b8-2fe142a2dfbe None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:17.807931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f66dd-7641-45a5-b8b8-2fe142a2dfbe None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:17.811804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c545a-5b63-43f6-9a00-ca124607f142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:17.812496 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c545a-5b63-43f6-9a00-ca124607f142 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:17.812909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c545a-5b63-43f6-9a00-ca124607f142 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:17.813103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c545a-5b63-43f6-9a00-ca124607f142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:17.815133 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-368c545a-5b63-43f6-9a00-ca124607f142 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:17.831484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c545a-5b63-43f6-9a00-ca124607f142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:17.833315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368c545a-5b63-43f6-9a00-ca124607f142 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:17.843677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1146/2264] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:17.952323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e4f54-df61-4c58-b2fb-68a0c21bced9 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:17.953825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f50e4f54-df61-4c58-b2fb-68a0c21bced9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:17.981449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e4f54-df61-4c58-b2fb-68a0c21bced9 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:17.986246 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d14e0-1719-434a-b38a-51fd0864c055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:17.986935 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d14e0-1719-434a-b38a-51fd0864c055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:17.987213 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d14e0-1719-434a-b38a-51fd0864c055 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:17.987483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d14e0-1719-434a-b38a-51fd0864c055 admin admin] PATH_INFO: `/v3/projects/a5da932c549c40f894932c2be69781aa` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:17.988663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d14e0-1719-434a-b38a-51fd0864c055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5da932c549c40f894932c2be69781aa)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:17.990718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d14e0-1719-434a-b38a-51fd0864c055 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:18.047577 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-192d14e0-1719-434a-b38a-51fd0864c055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5da932c549c40f894932c2be69781aa', 'request_id': 'req-192d14e0-1719-434a-b38a-51fd0864c055'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:18.054526 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-192d14e0-1719-434a-b38a-51fd0864c055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5da932c549c40f894932c2be69781aa', 'request_id': 'req-192d14e0-1719-434a-b38a-51fd0864c055'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:18.063798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-192d14e0-1719-434a-b38a-51fd0864c055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5da932c549c40f894932c2be69781aa', 'request_id': 'req-192d14e0-1719-434a-b38a-51fd0864c055'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:18.082040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-192d14e0-1719-434a-b38a-51fd0864c055 admin admin] The token cache is being invalidate because project a5da932c549c40f894932c2be69781aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:55:18.082443 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-192d14e0-1719-434a-b38a-51fd0864c055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:18.083187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-192d14e0-1719-434a-b38a-51fd0864c055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:55:18.084766 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1119/2265] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:55:17 2022] DELETE /identity/v3/projects/a5da932c549c40f894932c2be69781aa => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:55:18.089923 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ed5b3-5885-49e7-b7fb-ec79cfd2f04e None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:18.092273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ed5b3-5885-49e7-b7fb-ec79cfd2f04e None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:18.092273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ed5b3-5885-49e7-b7fb-ec79cfd2f04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:18.110800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e30ed5b3-5885-49e7-b7fb-ec79cfd2f04e None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:18.112039 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e30ed5b3-5885-49e7-b7fb-ec79cfd2f04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:18.186690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1147/2266] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:18.193343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe410a5-b4d6-49fe-aeea-a52179952ca8 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:18.193724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe410a5-b4d6-49fe-aeea-a52179952ca8 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:18.194075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe410a5-b4d6-49fe-aeea-a52179952ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:18.212701 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ed5b3-5885-49e7-b7fb-ec79cfd2f04e None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:18.214968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e30ed5b3-5885-49e7-b7fb-ec79cfd2f04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:18.223844 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-bbe410a5-b4d6-49fe-aeea-a52179952ca8 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:18.223844 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-bbe410a5-b4d6-49fe-aeea-a52179952ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:18.231280 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1120/2267] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:18.236595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe410a5-b4d6-49fe-aeea-a52179952ca8 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:18.248068 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe410a5-b4d6-49fe-aeea-a52179952ca8 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:18.254715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9bd40-e686-4f3f-9606-f9086466a3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:18.254715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9bd40-e686-4f3f-9606-f9086466a3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:18.254715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9bd40-e686-4f3f-9606-f9086466a3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:18.254715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9bd40-e686-4f3f-9606-f9086466a3f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:18.255158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9bd40-e686-4f3f-9606-f9086466a3f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:55:18.255357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9bd40-e686-4f3f-9606-f9086466a3f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:18.257287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9bd40-e686-4f3f-9606-f9086466a3f2 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:18.273523 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:18.280036 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1121/2268] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:55:18.285798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9bd40-e686-4f3f-9606-f9086466a3f2 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:18.296470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9bd40-e686-4f3f-9606-f9086466a3f2 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:18.307449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa23da-8ec9-42a2-a73a-629f6d3ac620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:18.308355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa23da-8ec9-42a2-a73a-629f6d3ac620 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:18.308788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa23da-8ec9-42a2-a73a-629f6d3ac620 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:18.309239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa23da-8ec9-42a2-a73a-629f6d3ac620 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:18.309995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa23da-8ec9-42a2-a73a-629f6d3ac620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:18.312839 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa23da-8ec9-42a2-a73a-629f6d3ac620 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:18.324272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ed5b3-5885-49e7-b7fb-ec79cfd2f04e None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:18.329621 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238403d-c605-49cf-a5dd-bec4afdd3325 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:18.330250 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238403d-c605-49cf-a5dd-bec4afdd3325 service placement] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:18.330525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238403d-c605-49cf-a5dd-bec4afdd3325 service placement] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:18.330839 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238403d-c605-49cf-a5dd-bec4afdd3325 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:18.332647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4238403d-c605-49cf-a5dd-bec4afdd3325 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:18.346449 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238403d-c605-49cf-a5dd-bec4afdd3325 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:18.348128 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238403d-c605-49cf-a5dd-bec4afdd3325 service placement] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:18.353681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1122/2269] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:55:18 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:55:18.354937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa23da-8ec9-42a2-a73a-629f6d3ac620 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:18.360529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1148/2270] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:18.435526 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa23da-8ec9-42a2-a73a-629f6d3ac620 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:18.445049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5768759-0bf7-47e6-b211-28794391f7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:18.445049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5768759-0bf7-47e6-b211-28794391f7ec admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:18.445049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5768759-0bf7-47e6-b211-28794391f7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:18.445049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5768759-0bf7-47e6-b211-28794391f7ec admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:18.449014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5768759-0bf7-47e6-b211-28794391f7ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:18.450444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5768759-0bf7-47e6-b211-28794391f7ec admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:18.479247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1123/2271] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:55:18 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:55:18.484047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238403d-c605-49cf-a5dd-bec4afdd3325 service placement] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:18.497917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238403d-c605-49cf-a5dd-bec4afdd3325 service placement] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:18.502797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cb742-818f-49cf-9895-7935d315a833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:18.503700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cb742-818f-49cf-9895-7935d315a833 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:18.504055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cb742-818f-49cf-9895-7935d315a833 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:18.504331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cb742-818f-49cf-9895-7935d315a833 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:18.505074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cb742-818f-49cf-9895-7935d315a833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:18.506720 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cb742-818f-49cf-9895-7935d315a833 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:18.523523 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1149/2272] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:55:18.526269 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5768759-0bf7-47e6-b211-28794391f7ec admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:18.532202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:18.538663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5768759-0bf7-47e6-b211-28794391f7ec admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:18.545150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dcbede-727c-454c-be19-69dd251f1273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:18.545901 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcbede-727c-454c-be19-69dd251f1273 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:18.546087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcbede-727c-454c-be19-69dd251f1273 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:18.546396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcbede-727c-454c-be19-69dd251f1273 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:18.547349 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dcbede-727c-454c-be19-69dd251f1273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:18.548791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dcbede-727c-454c-be19-69dd251f1273 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:18.561580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1124/2273] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:55:18.565651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cb742-818f-49cf-9895-7935d315a833 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:18.581542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cb742-818f-49cf-9895-7935d315a833 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:18.593991 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d02d3-a19d-4af9-8a56-9d71ab60822c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:18.594842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d02d3-a19d-4af9-8a56-9d71ab60822c admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:18.595181 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d02d3-a19d-4af9-8a56-9d71ab60822c admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:18.595514 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d02d3-a19d-4af9-8a56-9d71ab60822c admin admin] PATH_INFO: `/v3/projects/c492b732353043b3bfe7c77daad9df4c/users/48caa6935f9a45b994460b9cb40a4dcf/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:18.612464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d02d3-a19d-4af9-8a56-9d71ab60822c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c492b732353043b3bfe7c77daad9df4c, user_id=48caa6935f9a45b994460b9cb40a4dcf, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:18.615231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d02d3-a19d-4af9-8a56-9d71ab60822c admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:18.631629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1150/2274] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:55:18 2022] PUT /identity/v3/projects/c492b732353043b3bfe7c77daad9df4c/users/48caa6935f9a45b994460b9cb40a4dcf/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:55:18.637338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c181191-088f-4286-9262-ece4833f3cbb None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:18.637630 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c181191-088f-4286-9262-ece4833f3cbb None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:18.637907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c181191-088f-4286-9262-ece4833f3cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:18.659381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-5c181191-088f-4286-9262-ece4833f3cbb None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:18.660711 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5c181191-088f-4286-9262-ece4833f3cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:18.739219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1125/2275] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:18.748939 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d02d3-a19d-4af9-8a56-9d71ab60822c admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:18.750973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-af1d02d3-a19d-4af9-8a56-9d71ab60822c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:18.835308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d02d3-a19d-4af9-8a56-9d71ab60822c admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:18.841268 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185f5d7-4c6a-4487-a459-fe7229368bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:18.841797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185f5d7-4c6a-4487-a459-fe7229368bdd service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:18.842079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185f5d7-4c6a-4487-a459-fe7229368bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:18.842352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185f5d7-4c6a-4487-a459-fe7229368bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:18.853336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185f5d7-4c6a-4487-a459-fe7229368bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:18.855219 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185f5d7-4c6a-4487-a459-fe7229368bdd service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:18.869445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1151/2276] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:20.796614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c181191-088f-4286-9262-ece4833f3cbb None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:20.798817 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5c181191-088f-4286-9262-ece4833f3cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:20.822048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c181191-088f-4286-9262-ece4833f3cbb None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:20.861491 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a04229-f72c-4b5a-ae72-a6fb5f71f21f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:20.862479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a04229-f72c-4b5a-ae72-a6fb5f71f21f service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:20.862970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a04229-f72c-4b5a-ae72-a6fb5f71f21f service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:20.863633 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a04229-f72c-4b5a-ae72-a6fb5f71f21f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:20.869712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e4a04229-f72c-4b5a-ae72-a6fb5f71f21f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:20.893033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a04229-f72c-4b5a-ae72-a6fb5f71f21f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:20.905575 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a04229-f72c-4b5a-ae72-a6fb5f71f21f service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:20.918606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1126/2277] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:21.704447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185f5d7-4c6a-4487-a459-fe7229368bdd service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:21.706023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8185f5d7-4c6a-4487-a459-fe7229368bdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:21.869392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185f5d7-4c6a-4487-a459-fe7229368bdd service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:21.882862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e3694-0738-4734-bcf7-387fdaa715ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:21.885037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3694-0738-4734-bcf7-387fdaa715ec service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:21.885037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3694-0738-4734-bcf7-387fdaa715ec service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:21.885413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e3694-0738-4734-bcf7-387fdaa715ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:21.894680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-640e3694-0738-4734-bcf7-387fdaa715ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:21.927142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e3694-0738-4734-bcf7-387fdaa715ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:21.929022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e3694-0738-4734-bcf7-387fdaa715ec service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:21.950116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1152/2278] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:23.863371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a04229-f72c-4b5a-ae72-a6fb5f71f21f service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:23.864974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e4a04229-f72c-4b5a-ae72-a6fb5f71f21f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:24.068011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a04229-f72c-4b5a-ae72-a6fb5f71f21f service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:24.076225 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ef635-23d1-4e4f-8941-93b2d73f0a30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:24.076796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ef635-23d1-4e4f-8941-93b2d73f0a30 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:24.077077 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ef635-23d1-4e4f-8941-93b2d73f0a30 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:24.077352 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ef635-23d1-4e4f-8941-93b2d73f0a30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:24.083094 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2b4ef635-23d1-4e4f-8941-93b2d73f0a30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:24.215050 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ef635-23d1-4e4f-8941-93b2d73f0a30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:24.216688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ef635-23d1-4e4f-8941-93b2d73f0a30 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:24.259913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1127/2279] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:24.386847 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7778af7-26d0-42cf-9164-9b7156f432e7 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:24.387304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7778af7-26d0-42cf-9164-9b7156f432e7 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:24.387304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7778af7-26d0-42cf-9164-9b7156f432e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:24.402447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:24.426731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e7778af7-26d0-42cf-9164-9b7156f432e7 None None] MFA Rules not processed for user `48caa6935f9a45b994460b9cb40a4dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:24.428200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e7778af7-26d0-42cf-9164-9b7156f432e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:24.550290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1153/2280] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:55:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:24.551188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ef635-23d1-4e4f-8941-93b2d73f0a30 service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:24.567345 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ef635-23d1-4e4f-8941-93b2d73f0a30 service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:24.574253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180101f-126c-4b96-b97e-2c583d8ffe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:24.574982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180101f-126c-4b96-b97e-2c583d8ffe17 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:24.575276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180101f-126c-4b96-b97e-2c583d8ffe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:24.576650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180101f-126c-4b96-b97e-2c583d8ffe17 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:24.577778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180101f-126c-4b96-b97e-2c583d8ffe17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:24.580893 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180101f-126c-4b96-b97e-2c583d8ffe17 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:24.639751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1128/2281] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:55:24 2022] POST /identity/v3/projects => generated 385 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:55:24.641516 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7778af7-26d0-42cf-9164-9b7156f432e7 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:24.658414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7778af7-26d0-42cf-9164-9b7156f432e7 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:24.766550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da12bf-6f21-4dc7-9bb7-065dea18a17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:24.768042 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da12bf-6f21-4dc7-9bb7-065dea18a17a admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:24.768754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da12bf-6f21-4dc7-9bb7-065dea18a17a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:24.769448 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da12bf-6f21-4dc7-9bb7-065dea18a17a admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:24.770604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da12bf-6f21-4dc7-9bb7-065dea18a17a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:24.775581 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da12bf-6f21-4dc7-9bb7-065dea18a17a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:24.814507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1154/2282] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:55:24 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:55:24.819981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180101f-126c-4b96-b97e-2c583d8ffe17 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:24.844319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180101f-126c-4b96-b97e-2c583d8ffe17 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:24.851330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c15dc2-f66a-45d9-9175-cac05a093370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:24.852534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c15dc2-f66a-45d9-9175-cac05a093370 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:24.852534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c15dc2-f66a-45d9-9175-cac05a093370 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:24.853169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c15dc2-f66a-45d9-9175-cac05a093370 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:24.853797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c15dc2-f66a-45d9-9175-cac05a093370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:24.855812 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c15dc2-f66a-45d9-9175-cac05a093370 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:24.862322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:24.873990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1129/2283] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:55:24.877392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da12bf-6f21-4dc7-9bb7-065dea18a17a admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:24.891740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da12bf-6f21-4dc7-9bb7-065dea18a17a admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:24.896867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf3036-5cbe-49e9-8a4f-596a3d1791c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:24.897548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf3036-5cbe-49e9-8a4f-596a3d1791c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:24.897831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf3036-5cbe-49e9-8a4f-596a3d1791c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:24.898107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf3036-5cbe-49e9-8a4f-596a3d1791c6 admin admin] PATH_INFO: `/v3/projects/f497ae2e1f2e43c7adb89a248b86d3c6/users/d7d93cad817c4f42b37c897f72ba59e0/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:24.918903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf3036-5cbe-49e9-8a4f-596a3d1791c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f497ae2e1f2e43c7adb89a248b86d3c6, user_id=d7d93cad817c4f42b37c897f72ba59e0, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:24.919371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf3036-5cbe-49e9-8a4f-596a3d1791c6 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:24.948973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1155/2284] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:55:24 2022] PUT /identity/v3/projects/f497ae2e1f2e43c7adb89a248b86d3c6/users/d7d93cad817c4f42b37c897f72ba59e0/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:55:24.952917 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c15dc2-f66a-45d9-9175-cac05a093370 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:24.964617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c15dc2-f66a-45d9-9175-cac05a093370 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:25.034200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8cd65-ef38-46c1-8b41-ea0e73a8455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:25.034848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8cd65-ef38-46c1-8b41-ea0e73a8455b admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:25.035135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8cd65-ef38-46c1-8b41-ea0e73a8455b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:25.035407 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8cd65-ef38-46c1-8b41-ea0e73a8455b admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:25.036100 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f8cd65-ef38-46c1-8b41-ea0e73a8455b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:25.037700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f8cd65-ef38-46c1-8b41-ea0e73a8455b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:25.048873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1130/2285] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:55:25.053238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf3036-5cbe-49e9-8a4f-596a3d1791c6 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:25.066326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf3036-5cbe-49e9-8a4f-596a3d1791c6 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:25.071796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22568b7-f0ea-42eb-9eb9-8d1e2e30737b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:25.072476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22568b7-f0ea-42eb-9eb9-8d1e2e30737b admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:25.072768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22568b7-f0ea-42eb-9eb9-8d1e2e30737b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:25.073044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22568b7-f0ea-42eb-9eb9-8d1e2e30737b admin admin] PATH_INFO: `/v3/projects/f497ae2e1f2e43c7adb89a248b86d3c6/users/d7d93cad817c4f42b37c897f72ba59e0/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:25.075202 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22568b7-f0ea-42eb-9eb9-8d1e2e30737b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f497ae2e1f2e43c7adb89a248b86d3c6, user_id=d7d93cad817c4f42b37c897f72ba59e0, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:25.078022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22568b7-f0ea-42eb-9eb9-8d1e2e30737b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:25.089125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1156/2286] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:55:25 2022] PUT /identity/v3/projects/f497ae2e1f2e43c7adb89a248b86d3c6/users/d7d93cad817c4f42b37c897f72ba59e0/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:55:30.660375 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d050024-7382-463f-a3e9-f423ebc5692a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:30.661292 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d050024-7382-463f-a3e9-f423ebc5692a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:30.661786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d050024-7382-463f-a3e9-f423ebc5692a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:30.714970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-6d050024-7382-463f-a3e9-f423ebc5692a None None] MFA Rules not processed for user `d7d93cad817c4f42b37c897f72ba59e0`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:30.716610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6d050024-7382-463f-a3e9-f423ebc5692a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:30.853530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1131/2287] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:55:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:30.873271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22568b7-f0ea-42eb-9eb9-8d1e2e30737b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:30.876505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f22568b7-f0ea-42eb-9eb9-8d1e2e30737b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:30.941675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:30.949197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22568b7-f0ea-42eb-9eb9-8d1e2e30737b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:30.954470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48bd34b-b07a-4eec-9d3a-2a3257878722 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72d4459164844e718544d7fde333db76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:30.955599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bd34b-b07a-4eec-9d3a-2a3257878722 service glance] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:30.955947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bd34b-b07a-4eec-9d3a-2a3257878722 service glance] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:30.956717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48bd34b-b07a-4eec-9d3a-2a3257878722 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:31.050776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48bd34b-b07a-4eec-9d3a-2a3257878722 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:31.050776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48bd34b-b07a-4eec-9d3a-2a3257878722 service glance] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:31.064157 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1157/2288] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:31.106439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d050024-7382-463f-a3e9-f423ebc5692a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:31.108325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6d050024-7382-463f-a3e9-f423ebc5692a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:31.256015 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195897d-a207-4427-808b-852bfbc5f8d9 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:31.256304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195897d-a207-4427-808b-852bfbc5f8d9 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:31.256599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195897d-a207-4427-808b-852bfbc5f8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:31.302256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-d195897d-a207-4427-808b-852bfbc5f8d9 None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:31.304235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d195897d-a207-4427-808b-852bfbc5f8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:31.319093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d050024-7382-463f-a3e9-f423ebc5692a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:31.347380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c1454-27e7-4ea5-a69b-ec7dde113211 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:31.347826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c1454-27e7-4ea5-a69b-ec7dde113211 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:31.348103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c1454-27e7-4ea5-a69b-ec7dde113211 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:31.348370 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c1454-27e7-4ea5-a69b-ec7dde113211 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:31.364738 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-146c1454-27e7-4ea5-a69b-ec7dde113211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:31.401884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c1454-27e7-4ea5-a69b-ec7dde113211 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:31.410470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1158/2289] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:55:31.410780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c1454-27e7-4ea5-a69b-ec7dde113211 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:31.435884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1132/2290] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:31.516385 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195897d-a207-4427-808b-852bfbc5f8d9 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:31.518503 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d195897d-a207-4427-808b-852bfbc5f8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:31.650776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195897d-a207-4427-808b-852bfbc5f8d9 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:31.657909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0701099-1435-43c0-9e2e-f0ac737fe6a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:31.658843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0701099-1435-43c0-9e2e-f0ac737fe6a6 service swift] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:31.659196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0701099-1435-43c0-9e2e-f0ac737fe6a6 service swift] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:31.659589 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0701099-1435-43c0-9e2e-f0ac737fe6a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:31.669159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0701099-1435-43c0-9e2e-f0ac737fe6a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:31.670993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0701099-1435-43c0-9e2e-f0ac737fe6a6 service swift] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:31.689172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1159/2291] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:55:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:55:38.872695 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c1454-27e7-4ea5-a69b-ec7dde113211 service nova] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:38.884943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c1454-27e7-4ea5-a69b-ec7dde113211 service nova] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:38.892028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1e1ee-5d7a-41a9-b329-c6716037c124 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:38.892652 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1e1ee-5d7a-41a9-b329-c6716037c124 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:38.892930 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1e1ee-5d7a-41a9-b329-c6716037c124 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:38.893201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1e1ee-5d7a-41a9-b329-c6716037c124 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:38.895037 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-62c1e1ee-5d7a-41a9-b329-c6716037c124 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:38.900818 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:38.975007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1e1ee-5d7a-41a9-b329-c6716037c124 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:38.977365 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1e1ee-5d7a-41a9-b329-c6716037c124 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:38.989898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1133/2292] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:55:41.677464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=84 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:43.070287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6a7c5-b478-4c21-8afb-73d00ba30465 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:43.070970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6a7c5-b478-4c21-8afb-73d00ba30465 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:43.071237 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6a7c5-b478-4c21-8afb-73d00ba30465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:43.091190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-b8d6a7c5-b478-4c21-8afb-73d00ba30465 None None] MFA Rules not processed for user `48caa6935f9a45b994460b9cb40a4dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:43.092392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b8d6a7c5-b478-4c21-8afb-73d00ba30465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:43.099783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1160/2293] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:55:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:46.389805 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69614856-aa94-48cf-9c96-3ed3b25b0fab None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:46.390310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69614856-aa94-48cf-9c96-3ed3b25b0fab None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:46.390392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69614856-aa94-48cf-9c96-3ed3b25b0fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:46.415551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-69614856-aa94-48cf-9c96-3ed3b25b0fab None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:55:46.418228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-69614856-aa94-48cf-9c96-3ed3b25b0fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:46.446842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1134/2294] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:55:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:55:46.448787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6a7c5-b478-4c21-8afb-73d00ba30465 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:46.473077 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6a7c5-b478-4c21-8afb-73d00ba30465 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:46.481437 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b8403-01ed-420e-9e7d-8d98fc0f5bb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:46.482064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b8403-01ed-420e-9e7d-8d98fc0f5bb9 service swift] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:46.482455 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b8403-01ed-420e-9e7d-8d98fc0f5bb9 service swift] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:46.482612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b8403-01ed-420e-9e7d-8d98fc0f5bb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:46.492296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b8403-01ed-420e-9e7d-8d98fc0f5bb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:46.494085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b8403-01ed-420e-9e7d-8d98fc0f5bb9 service swift] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:46.508983 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1161/2295] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:55:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:55:56.407716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=28 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:56.502527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=9 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:55:58.012750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69614856-aa94-48cf-9c96-3ed3b25b0fab None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:55:58.072133 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69614856-aa94-48cf-9c96-3ed3b25b0fab None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:55:58.075904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b138d4-3808-4e7f-8065-e37f44a6bd97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:55:58.076520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b138d4-3808-4e7f-8065-e37f44a6bd97 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:55:58.076796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b138d4-3808-4e7f-8065-e37f44a6bd97 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:55:58.077093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b138d4-3808-4e7f-8065-e37f44a6bd97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:55:58.079321 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-70b138d4-3808-4e7f-8065-e37f44a6bd97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:55:58.146407 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b138d4-3808-4e7f-8065-e37f44a6bd97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:55:58.148049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b138d4-3808-4e7f-8065-e37f44a6bd97 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:55:58.158134 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1135/2296] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:01.995412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b8403-01ed-420e-9e7d-8d98fc0f5bb9 service swift] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:02.083900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b8403-01ed-420e-9e7d-8d98fc0f5bb9 service swift] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:02.098873 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c77d5-4907-4aec-a0a0-636c3d92d8b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:02.099536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c77d5-4907-4aec-a0a0-636c3d92d8b5 service placement] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:02.099836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c77d5-4907-4aec-a0a0-636c3d92d8b5 service placement] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:02.101340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c77d5-4907-4aec-a0a0-636c3d92d8b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:02.102933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f02c77d5-4907-4aec-a0a0-636c3d92d8b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:02.117442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c77d5-4907-4aec-a0a0-636c3d92d8b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:02.130835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c77d5-4907-4aec-a0a0-636c3d92d8b5 service placement] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:02.130835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1162/2297] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:02.193020 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b138d4-3808-4e7f-8065-e37f44a6bd97 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:02.195427 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-70b138d4-3808-4e7f-8065-e37f44a6bd97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:02.212159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b138d4-3808-4e7f-8065-e37f44a6bd97 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:02.221422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0f3e9-1631-4b94-a6b3-e4240555a712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:02.222171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0f3e9-1631-4b94-a6b3-e4240555a712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:02.222782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0f3e9-1631-4b94-a6b3-e4240555a712 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:02.223138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0f3e9-1631-4b94-a6b3-e4240555a712 admin admin] PATH_INFO: `/v3/users/d183fcdae86448caa81c09823dd3884e` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:02.224668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0f3e9-1631-4b94-a6b3-e4240555a712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d183fcdae86448caa81c09823dd3884e)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:02.228814 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0f3e9-1631-4b94-a6b3-e4240555a712 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:02.288629 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f0c0f3e9-1631-4b94-a6b3-e4240555a712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd183fcdae86448caa81c09823dd3884e'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:02.296529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f0c0f3e9-1631-4b94-a6b3-e4240555a712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd183fcdae86448caa81c09823dd3884e'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:02.297238 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f0c0f3e9-1631-4b94-a6b3-e4240555a712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd183fcdae86448caa81c09823dd3884e'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:02.310454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f0c0f3e9-1631-4b94-a6b3-e4240555a712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd183fcdae86448caa81c09823dd3884e'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:02.318661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f0c0f3e9-1631-4b94-a6b3-e4240555a712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd183fcdae86448caa81c09823dd3884e'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:02.325900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1136/2298] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:56:02 2022] DELETE /identity/v3/users/d183fcdae86448caa81c09823dd3884e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:02.692559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c77d5-4907-4aec-a0a0-636c3d92d8b5 service placement] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:02.716526 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c77d5-4907-4aec-a0a0-636c3d92d8b5 service placement] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:02.818599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86433a55-c986-47e2-ba0e-c36f1ed39224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:02.819791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86433a55-c986-47e2-ba0e-c36f1ed39224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:02.820062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86433a55-c986-47e2-ba0e-c36f1ed39224 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:02.820325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86433a55-c986-47e2-ba0e-c36f1ed39224 admin admin] PATH_INFO: `/v3/projects/afed1554d1394cfc8abea0e6c5e15b3f` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:02.821515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86433a55-c986-47e2-ba0e-c36f1ed39224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afed1554d1394cfc8abea0e6c5e15b3f)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:02.823560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86433a55-c986-47e2-ba0e-c36f1ed39224 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:02.879403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-86433a55-c986-47e2-ba0e-c36f1ed39224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afed1554d1394cfc8abea0e6c5e15b3f', 'request_id': 'req-86433a55-c986-47e2-ba0e-c36f1ed39224'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:02.886870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-86433a55-c986-47e2-ba0e-c36f1ed39224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afed1554d1394cfc8abea0e6c5e15b3f', 'request_id': 'req-86433a55-c986-47e2-ba0e-c36f1ed39224'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:02.894678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-86433a55-c986-47e2-ba0e-c36f1ed39224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afed1554d1394cfc8abea0e6c5e15b3f', 'request_id': 'req-86433a55-c986-47e2-ba0e-c36f1ed39224'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:02.907550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-86433a55-c986-47e2-ba0e-c36f1ed39224 admin admin] The token cache is being invalidate because project afed1554d1394cfc8abea0e6c5e15b3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:56:02.907831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-86433a55-c986-47e2-ba0e-c36f1ed39224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:02.908755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-86433a55-c986-47e2-ba0e-c36f1ed39224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:02.910277 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1163/2299] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:56:02 2022] DELETE /identity/v3/projects/afed1554d1394cfc8abea0e6c5e15b3f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:02.916458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6687e-93d6-457b-a3da-d7254b402e49 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:02.916965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6687e-93d6-457b-a3da-d7254b402e49 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:02.917453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6687e-93d6-457b-a3da-d7254b402e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:02.940949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-dee6687e-93d6-457b-a3da-d7254b402e49 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:02.942480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-dee6687e-93d6-457b-a3da-d7254b402e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:02.962836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:03.014438 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1137/2300] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:56:03.023691 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844866f3-055e-48dc-9bd8-99b95be2a17b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:03.024273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844866f3-055e-48dc-9bd8-99b95be2a17b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:03.024273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844866f3-055e-48dc-9bd8-99b95be2a17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:03.046228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-844866f3-055e-48dc-9bd8-99b95be2a17b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:03.047501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-844866f3-055e-48dc-9bd8-99b95be2a17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:03.056565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1164/2301] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:03.061288 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6687e-93d6-457b-a3da-d7254b402e49 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:03.072496 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6687e-93d6-457b-a3da-d7254b402e49 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:03.077111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e73aa8-d005-4381-ac43-0f74766dd742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:03.077734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e73aa8-d005-4381-ac43-0f74766dd742 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:03.078008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e73aa8-d005-4381-ac43-0f74766dd742 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:03.078276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e73aa8-d005-4381-ac43-0f74766dd742 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:03.079078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e73aa8-d005-4381-ac43-0f74766dd742 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:56:03.079373 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e73aa8-d005-4381-ac43-0f74766dd742 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:03.081158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e73aa8-d005-4381-ac43-0f74766dd742 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:03.095253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1138/2302] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:56:03.100058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844866f3-055e-48dc-9bd8-99b95be2a17b None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:03.111128 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844866f3-055e-48dc-9bd8-99b95be2a17b None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:03.116849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8957f-a1e2-4bdf-b3de-879b2912228e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:03.117647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8957f-a1e2-4bdf-b3de-879b2912228e admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:03.117931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8957f-a1e2-4bdf-b3de-879b2912228e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:03.118265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8957f-a1e2-4bdf-b3de-879b2912228e admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:03.119450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8957f-a1e2-4bdf-b3de-879b2912228e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:03.121732 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8957f-a1e2-4bdf-b3de-879b2912228e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:03.140656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:03.170773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1165/2303] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:56:03 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:56:03.175024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e73aa8-d005-4381-ac43-0f74766dd742 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:03.199366 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e73aa8-d005-4381-ac43-0f74766dd742 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:03.333258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbd7ba-8e92-4134-8e04-2aa28ce419da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:03.334083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbd7ba-8e92-4134-8e04-2aa28ce419da admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:03.334407 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbd7ba-8e92-4134-8e04-2aa28ce419da admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:03.334720 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbd7ba-8e92-4134-8e04-2aa28ce419da admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:03.335401 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbd7ba-8e92-4134-8e04-2aa28ce419da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:03.337357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbd7ba-8e92-4134-8e04-2aa28ce419da admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:03.376895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1139/2304] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:56:03 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:56:03.378181 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8957f-a1e2-4bdf-b3de-879b2912228e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:03.391941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8957f-a1e2-4bdf-b3de-879b2912228e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:03.396854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa47f51-4797-49d5-a7b2-31ba63a9b42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:03.398141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa47f51-4797-49d5-a7b2-31ba63a9b42f admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:03.398283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa47f51-4797-49d5-a7b2-31ba63a9b42f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:03.398979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa47f51-4797-49d5-a7b2-31ba63a9b42f admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:03.399224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa47f51-4797-49d5-a7b2-31ba63a9b42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:03.400876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa47f51-4797-49d5-a7b2-31ba63a9b42f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:03.418687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1166/2305] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:03.422906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbd7ba-8e92-4134-8e04-2aa28ce419da admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:03.436480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbd7ba-8e92-4134-8e04-2aa28ce419da admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:03.441696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fb3e3-26d5-4af2-84c1-665c89699a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:03.442371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fb3e3-26d5-4af2-84c1-665c89699a51 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:03.442596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fb3e3-26d5-4af2-84c1-665c89699a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:03.442902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fb3e3-26d5-4af2-84c1-665c89699a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:03.443636 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fb3e3-26d5-4af2-84c1-665c89699a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:03.445878 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fb3e3-26d5-4af2-84c1-665c89699a51 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:03.464695 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1140/2306] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:03.469629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa47f51-4797-49d5-a7b2-31ba63a9b42f admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:03.491974 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa47f51-4797-49d5-a7b2-31ba63a9b42f admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:03.504624 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bfb2a-cedc-433f-a82e-412d6edde7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:03.505295 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bfb2a-cedc-433f-a82e-412d6edde7cd admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:03.505579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bfb2a-cedc-433f-a82e-412d6edde7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:03.505940 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bfb2a-cedc-433f-a82e-412d6edde7cd admin admin] PATH_INFO: `/v3/projects/618b229691bb4875b775d74261fefb1d/users/162f406f06ea460e9531bfc72b6121bc/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:03.540502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bfb2a-cedc-433f-a82e-412d6edde7cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=618b229691bb4875b775d74261fefb1d, user_id=162f406f06ea460e9531bfc72b6121bc, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:03.545177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bfb2a-cedc-433f-a82e-412d6edde7cd admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:03.575271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1167/2307] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:56:03 2022] PUT /identity/v3/projects/618b229691bb4875b775d74261fefb1d/users/162f406f06ea460e9531bfc72b6121bc/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:03.581792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ceac6a-cf6f-488e-832e-9b6e7a40f573 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:03.582105 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ceac6a-cf6f-488e-832e-9b6e7a40f573 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:03.582341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ceac6a-cf6f-488e-832e-9b6e7a40f573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:03.613215 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-93ceac6a-cf6f-488e-832e-9b6e7a40f573 None None] MFA Rules not processed for user `162f406f06ea460e9531bfc72b6121bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:03.614487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-93ceac6a-cf6f-488e-832e-9b6e7a40f573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:03.688312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1141/2308] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:03.700018 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bfb2a-cedc-433f-a82e-412d6edde7cd admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:03.701766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d95bfb2a-cedc-433f-a82e-412d6edde7cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:03.777445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bfb2a-cedc-433f-a82e-412d6edde7cd admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:03.783534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ce768-d083-4176-a2e0-5016f51b2e50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:03.784090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ce768-d083-4176-a2e0-5016f51b2e50 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:03.784366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ce768-d083-4176-a2e0-5016f51b2e50 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:03.784636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ce768-d083-4176-a2e0-5016f51b2e50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:03.795285 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ce768-d083-4176-a2e0-5016f51b2e50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:03.796999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ce768-d083-4176-a2e0-5016f51b2e50 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:03.822954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1168/2309] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:04.239303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ceac6a-cf6f-488e-832e-9b6e7a40f573 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:04.268663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ceac6a-cf6f-488e-832e-9b6e7a40f573 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:04.276210 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa6cd6-7f47-4de8-b850-b95366d66a23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:04.276819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa6cd6-7f47-4de8-b850-b95366d66a23 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:04.277082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa6cd6-7f47-4de8-b850-b95366d66a23 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:04.277564 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa6cd6-7f47-4de8-b850-b95366d66a23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:04.279693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d0fa6cd6-7f47-4de8-b850-b95366d66a23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:04.294389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa6cd6-7f47-4de8-b850-b95366d66a23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:04.296026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa6cd6-7f47-4de8-b850-b95366d66a23 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:04.308483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1142/2310] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:06.673874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ce768-d083-4176-a2e0-5016f51b2e50 service nova] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:06.739298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ce768-d083-4176-a2e0-5016f51b2e50 service nova] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:06.744879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:06.745515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:06.745789 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:06.746050 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] PATH_INFO: `/v3/users/162f406f06ea460e9531bfc72b6121bc` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:06.748899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=162f406f06ea460e9531bfc72b6121bc)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:06.749574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:06.804502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:06.807585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162f406f06ea460e9531bfc72b6121bc'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:06.824838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162f406f06ea460e9531bfc72b6121bc'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:06.836471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162f406f06ea460e9531bfc72b6121bc'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:06.837131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162f406f06ea460e9531bfc72b6121bc'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:06.837735 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162f406f06ea460e9531bfc72b6121bc'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:06.859046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1169/2311] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:56:06 2022] DELETE /identity/v3/users/162f406f06ea460e9531bfc72b6121bc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:06.861073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0441fb-1012-4931-a3e3-ca01cc2d6019 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:06.861798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0441fb-1012-4931-a3e3-ca01cc2d6019 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:06.861798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0441fb-1012-4931-a3e3-ca01cc2d6019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:06.876123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:06.901404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-1c0441fb-1012-4931-a3e3-ca01cc2d6019 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:06.903103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-1c0441fb-1012-4931-a3e3-ca01cc2d6019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:07.016809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1143/2312] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:07.048294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:07.050235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:07.123299 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035e404-7850-491d-9f57-932afd567f11 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:07.128500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec91d2-c721-4969-a7b0-0931453292bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:07.129221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec91d2-c721-4969-a7b0-0931453292bb service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:07.129559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec91d2-c721-4969-a7b0-0931453292bb service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:07.129893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec91d2-c721-4969-a7b0-0931453292bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:07.141254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec91d2-c721-4969-a7b0-0931453292bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:07.142927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec91d2-c721-4969-a7b0-0931453292bb service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:07.158245 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1170/2313] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:07.464052 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8f486-d05a-48ba-9b9a-45c52942d25d None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:07.464335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8f486-d05a-48ba-9b9a-45c52942d25d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:07.464826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8f486-d05a-48ba-9b9a-45c52942d25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:07.489999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-b1c8f486-d05a-48ba-9b9a-45c52942d25d None None] MFA Rules not processed for user `a357a3d27e4e41adae1c56ee39ae5aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:07.492262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b1c8f486-d05a-48ba-9b9a-45c52942d25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:07.576720 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1144/2314] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:07.633457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec91d2-c721-4969-a7b0-0931453292bb service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:07.638099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4cec91d2-c721-4969-a7b0-0931453292bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:07.662950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec91d2-c721-4969-a7b0-0931453292bb service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:07.685216 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:07.685973 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:07.686317 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:07.686685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8 admin admin] PATH_INFO: `/v3/projects/618b229691bb4875b775d74261fefb1d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:07.688113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=618b229691bb4875b775d74261fefb1d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:07.690201 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:07.754133 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '618b229691bb4875b775d74261fefb1d', 'request_id': 'req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:07.761863 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '618b229691bb4875b775d74261fefb1d', 'request_id': 'req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:07.772779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '618b229691bb4875b775d74261fefb1d', 'request_id': 'req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:07.788070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8 admin admin] The token cache is being invalidate because project 618b229691bb4875b775d74261fefb1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:56:07.788416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:07.788972 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-6d4fa8fa-dc8e-4dc0-835f-157d7aa1f4f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:07.790530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1171/2315] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:56:07 2022] DELETE /identity/v3/projects/618b229691bb4875b775d74261fefb1d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:07.797952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dfc1b-1aef-45d3-a59d-c3517922818d None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:07.798429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dfc1b-1aef-45d3-a59d-c3517922818d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:07.799189 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dfc1b-1aef-45d3-a59d-c3517922818d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:07.831653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-b13dfc1b-1aef-45d3-a59d-c3517922818d None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:07.834086 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b13dfc1b-1aef-45d3-a59d-c3517922818d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:07.946980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1145/2316] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:07.950983 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3017b-0a1b-41f4-b960-88edbe73d18b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:07.951343 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3017b-0a1b-41f4-b960-88edbe73d18b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:07.951707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3017b-0a1b-41f4-b960-88edbe73d18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:07.973675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-b6e3017b-0a1b-41f4-b960-88edbe73d18b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:07.977048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b6e3017b-0a1b-41f4-b960-88edbe73d18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:07.993539 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1172/2317] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:07.999203 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dfc1b-1aef-45d3-a59d-c3517922818d None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:08.012026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dfc1b-1aef-45d3-a59d-c3517922818d None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:08.016975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3017b-0a1b-41f4-b960-88edbe73d18b None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:08.017981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb15ae-78c4-4276-abaa-b306847003fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:08.018877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb15ae-78c4-4276-abaa-b306847003fa admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:08.019156 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb15ae-78c4-4276-abaa-b306847003fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:08.019428 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb15ae-78c4-4276-abaa-b306847003fa admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:08.020134 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb15ae-78c4-4276-abaa-b306847003fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:56:08.020437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb15ae-78c4-4276-abaa-b306847003fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:08.021300 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b6e3017b-0a1b-41f4-b960-88edbe73d18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:08.022508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb15ae-78c4-4276-abaa-b306847003fa admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:08.041203 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1146/2318] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:56:08.044987 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb15ae-78c4-4276-abaa-b306847003fa admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:08.058845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb15ae-78c4-4276-abaa-b306847003fa admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:08.065851 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08181a02-2bb0-40f1-bf05-5e98cde1b19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:08.067132 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08181a02-2bb0-40f1-bf05-5e98cde1b19f admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:08.067921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08181a02-2bb0-40f1-bf05-5e98cde1b19f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:08.069130 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08181a02-2bb0-40f1-bf05-5e98cde1b19f admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:08.070004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08181a02-2bb0-40f1-bf05-5e98cde1b19f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:08.072981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08181a02-2bb0-40f1-bf05-5e98cde1b19f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:08.093844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:08.110556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1147/2319] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:56:08 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:56:08.112269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3017b-0a1b-41f4-b960-88edbe73d18b None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:08.115377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08181a02-2bb0-40f1-bf05-5e98cde1b19f admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:08.136732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:08.172824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b21ca9-b52b-4812-a5d7-4cbf5d1dbed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:08.173479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b21ca9-b52b-4812-a5d7-4cbf5d1dbed9 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:08.173756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b21ca9-b52b-4812-a5d7-4cbf5d1dbed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:08.174025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b21ca9-b52b-4812-a5d7-4cbf5d1dbed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:08.175862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-98b21ca9-b52b-4812-a5d7-4cbf5d1dbed9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:08.186806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b21ca9-b52b-4812-a5d7-4cbf5d1dbed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:08.188247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b21ca9-b52b-4812-a5d7-4cbf5d1dbed9 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:08.194811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08181a02-2bb0-40f1-bf05-5e98cde1b19f admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:08.197030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1173/2320] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:08.200969 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f75349-8aff-41b7-8bfd-612debfb2d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:08.201607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f75349-8aff-41b7-8bfd-612debfb2d65 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:08.201879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f75349-8aff-41b7-8bfd-612debfb2d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:08.202153 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f75349-8aff-41b7-8bfd-612debfb2d65 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:08.203063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f75349-8aff-41b7-8bfd-612debfb2d65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:08.206676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f75349-8aff-41b7-8bfd-612debfb2d65 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:08.238430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1148/2321] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:56:08 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:56:08.239690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b21ca9-b52b-4812-a5d7-4cbf5d1dbed9 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:08.264205 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b21ca9-b52b-4812-a5d7-4cbf5d1dbed9 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:08.270571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01985583-031e-4be8-a50e-1ad16bb0d30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:08.271246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01985583-031e-4be8-a50e-1ad16bb0d30c admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:08.272927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01985583-031e-4be8-a50e-1ad16bb0d30c admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:08.273230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01985583-031e-4be8-a50e-1ad16bb0d30c admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:08.273937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01985583-031e-4be8-a50e-1ad16bb0d30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:08.275574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01985583-031e-4be8-a50e-1ad16bb0d30c admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:08.286847 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1174/2322] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:08.291887 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f75349-8aff-41b7-8bfd-612debfb2d65 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:08.302067 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f75349-8aff-41b7-8bfd-612debfb2d65 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:08.307792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f66932-da00-408d-8668-395a17c65a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:08.308893 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f66932-da00-408d-8668-395a17c65a76 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:08.309514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f66932-da00-408d-8668-395a17c65a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:08.309652 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f66932-da00-408d-8668-395a17c65a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:08.310464 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f66932-da00-408d-8668-395a17c65a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:08.312280 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f66932-da00-408d-8668-395a17c65a76 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:08.326945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1149/2323] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:08.331190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01985583-031e-4be8-a50e-1ad16bb0d30c admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:08.344450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01985583-031e-4be8-a50e-1ad16bb0d30c admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:08.351570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b44b4a-baea-4b47-a064-4df1988f2fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:08.352828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b44b4a-baea-4b47-a064-4df1988f2fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:08.353225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b44b4a-baea-4b47-a064-4df1988f2fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:08.353567 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b44b4a-baea-4b47-a064-4df1988f2fa3 admin admin] PATH_INFO: `/v3/projects/b0a9eed9af9a46d7a9c7512640bdd1b7/users/73cf1577d89342d99334f47016e7be71/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:08.370953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b44b4a-baea-4b47-a064-4df1988f2fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0a9eed9af9a46d7a9c7512640bdd1b7, user_id=73cf1577d89342d99334f47016e7be71, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:08.374199 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b44b4a-baea-4b47-a064-4df1988f2fa3 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:08.400123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1175/2324] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:56:08 2022] PUT /identity/v3/projects/b0a9eed9af9a46d7a9c7512640bdd1b7/users/73cf1577d89342d99334f47016e7be71/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:56:08.405511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b868d6-81e5-4780-a647-cf2ab53d1a8f None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:08.405801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b868d6-81e5-4780-a647-cf2ab53d1a8f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:08.406070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b868d6-81e5-4780-a647-cf2ab53d1a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:08.443750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-15b868d6-81e5-4780-a647-cf2ab53d1a8f None None] MFA Rules not processed for user `73cf1577d89342d99334f47016e7be71`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:08.446975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-15b868d6-81e5-4780-a647-cf2ab53d1a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:08.530351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1150/2325] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:08.535881 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b44b4a-baea-4b47-a064-4df1988f2fa3 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:08.641154 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b44b4a-baea-4b47-a064-4df1988f2fa3 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:08.655107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93fc31-668d-4d28-b8f2-0794ff7ca0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:08.655904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93fc31-668d-4d28-b8f2-0794ff7ca0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:08.656696 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93fc31-668d-4d28-b8f2-0794ff7ca0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:08.656696 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93fc31-668d-4d28-b8f2-0794ff7ca0d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:08.657646 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93fc31-668d-4d28-b8f2-0794ff7ca0d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:08.659027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93fc31-668d-4d28-b8f2-0794ff7ca0d6 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:08.697571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1176/2326] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:56:08 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:56:08.699438 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b868d6-81e5-4780-a647-cf2ab53d1a8f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:08.711443 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b868d6-81e5-4780-a647-cf2ab53d1a8f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:08.789253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b030d-b6d2-4b74-b14e-b2a56aad2a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:08.790200 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b030d-b6d2-4b74-b14e-b2a56aad2a30 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:08.790721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b030d-b6d2-4b74-b14e-b2a56aad2a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:08.791204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b030d-b6d2-4b74-b14e-b2a56aad2a30 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:08.792918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b030d-b6d2-4b74-b14e-b2a56aad2a30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:08.797214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b030d-b6d2-4b74-b14e-b2a56aad2a30 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:08.827624 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1151/2327] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:56:08 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:56:08.832676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93fc31-668d-4d28-b8f2-0794ff7ca0d6 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:08.848231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93fc31-668d-4d28-b8f2-0794ff7ca0d6 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:08.854530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c33a52-e49d-495b-a06d-31c9b878de78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:08.855188 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c33a52-e49d-495b-a06d-31c9b878de78 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:08.855464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c33a52-e49d-495b-a06d-31c9b878de78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:08.855733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c33a52-e49d-495b-a06d-31c9b878de78 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:08.856468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c33a52-e49d-495b-a06d-31c9b878de78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:08.858032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c33a52-e49d-495b-a06d-31c9b878de78 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:08.871423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1177/2328] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:08.875635 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b030d-b6d2-4b74-b14e-b2a56aad2a30 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:08.886976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b030d-b6d2-4b74-b14e-b2a56aad2a30 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:08.893207 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b315b-2eb8-4427-95a3-310f0da92985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:08.894166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b315b-2eb8-4427-95a3-310f0da92985 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:08.894654 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b315b-2eb8-4427-95a3-310f0da92985 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:08.894944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b315b-2eb8-4427-95a3-310f0da92985 admin admin] PATH_INFO: `/v3/projects/7b2fc6d0215c4e08b40f49f4b5248b63/users/6d53b5590d9140c08768811044a34504/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:08.909881 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:08.917828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b315b-2eb8-4427-95a3-310f0da92985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2fc6d0215c4e08b40f49f4b5248b63, user_id=6d53b5590d9140c08768811044a34504, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:08.920641 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b315b-2eb8-4427-95a3-310f0da92985 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:08.939128 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1152/2329] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:56:08 2022] PUT /identity/v3/projects/7b2fc6d0215c4e08b40f49f4b5248b63/users/6d53b5590d9140c08768811044a34504/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:56:08.943988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c33a52-e49d-495b-a06d-31c9b878de78 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:08.955896 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c33a52-e49d-495b-a06d-31c9b878de78 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:09.012417 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:09.047020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d9694-2bf2-45e3-8c6c-ef8014062174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:09.047550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d9694-2bf2-45e3-8c6c-ef8014062174 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:09.047825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d9694-2bf2-45e3-8c6c-ef8014062174 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:09.048095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d9694-2bf2-45e3-8c6c-ef8014062174 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:09.048819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d9694-2bf2-45e3-8c6c-ef8014062174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:09.050460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d9694-2bf2-45e3-8c6c-ef8014062174 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:09.074776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1178/2330] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:56:09.079867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b315b-2eb8-4427-95a3-310f0da92985 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:09.091482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b315b-2eb8-4427-95a3-310f0da92985 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:09.098480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d07b8f-1aa2-4b3b-976a-6ef7ab5dc0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:09.099796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d07b8f-1aa2-4b3b-976a-6ef7ab5dc0ea admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:09.100081 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d07b8f-1aa2-4b3b-976a-6ef7ab5dc0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:09.101163 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d07b8f-1aa2-4b3b-976a-6ef7ab5dc0ea admin admin] PATH_INFO: `/v3/projects/7b2fc6d0215c4e08b40f49f4b5248b63/users/6d53b5590d9140c08768811044a34504/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:09.103979 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d07b8f-1aa2-4b3b-976a-6ef7ab5dc0ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2fc6d0215c4e08b40f49f4b5248b63, user_id=6d53b5590d9140c08768811044a34504, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:09.107269 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d07b8f-1aa2-4b3b-976a-6ef7ab5dc0ea admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:09.138322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1153/2331] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:56:09 2022] PUT /identity/v3/projects/7b2fc6d0215c4e08b40f49f4b5248b63/users/6d53b5590d9140c08768811044a34504/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:09.148328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb2ecb-de95-4fd7-9fc5-fa29beeda203 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:09.148665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb2ecb-de95-4fd7-9fc5-fa29beeda203 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:09.148970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb2ecb-de95-4fd7-9fc5-fa29beeda203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:09.181557 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-1cbb2ecb-de95-4fd7-9fc5-fa29beeda203 None None] MFA Rules not processed for user `6d53b5590d9140c08768811044a34504`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:09.182798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1cbb2ecb-de95-4fd7-9fc5-fa29beeda203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:09.276547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1179/2332] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:09.295683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d07b8f-1aa2-4b3b-976a-6ef7ab5dc0ea admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:09.298668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-69d07b8f-1aa2-4b3b-976a-6ef7ab5dc0ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:09.399533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d07b8f-1aa2-4b3b-976a-6ef7ab5dc0ea admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:09.404774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652b096-31a8-43fb-a9d9-cd2bf38fba11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:09.405535 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652b096-31a8-43fb-a9d9-cd2bf38fba11 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:09.405906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652b096-31a8-43fb-a9d9-cd2bf38fba11 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:09.406282 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652b096-31a8-43fb-a9d9-cd2bf38fba11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:09.421219 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652b096-31a8-43fb-a9d9-cd2bf38fba11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:09.422759 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652b096-31a8-43fb-a9d9-cd2bf38fba11 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:09.435877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1154/2333] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:09.469619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb2ecb-de95-4fd7-9fc5-fa29beeda203 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:09.565867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb2ecb-de95-4fd7-9fc5-fa29beeda203 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:09.573352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:09.574532 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:09.574981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:09.575433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] PATH_INFO: `/v3/users/73cf1577d89342d99334f47016e7be71` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:09.577327 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73cf1577d89342d99334f47016e7be71)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:09.580508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:09.713067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73cf1577d89342d99334f47016e7be71'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:09.735149 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:09.737935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73cf1577d89342d99334f47016e7be71'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:09.754439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73cf1577d89342d99334f47016e7be71'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:09.756817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73cf1577d89342d99334f47016e7be71'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:09.757606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73cf1577d89342d99334f47016e7be71'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:09.795331 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc36b52-9246-4b11-86c9-2195387b1c6e None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:09.795777 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc36b52-9246-4b11-86c9-2195387b1c6e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:09.796173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc36b52-9246-4b11-86c9-2195387b1c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:09.799333 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1180/2334] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:56:09 2022] DELETE /identity/v3/users/73cf1577d89342d99334f47016e7be71 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:09.821857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-4dc36b52-9246-4b11-86c9-2195387b1c6e None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:09.823682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-4dc36b52-9246-4b11-86c9-2195387b1c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:09.949945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1155/2335] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:09.961484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:09.963115 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:10.070649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c8a06-e3a8-4ae4-8ede-17ee9dc0cff2 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:10.075273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09038641-d07e-4dc0-98f7-6e01c89d152e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:10.075998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09038641-d07e-4dc0-98f7-6e01c89d152e service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:10.076313 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09038641-d07e-4dc0-98f7-6e01c89d152e service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:10.076585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09038641-d07e-4dc0-98f7-6e01c89d152e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:10.089336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09038641-d07e-4dc0-98f7-6e01c89d152e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:10.091891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09038641-d07e-4dc0-98f7-6e01c89d152e service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:10.111373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1181/2336] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:10.646460 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc36b52-9246-4b11-86c9-2195387b1c6e None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:10.648941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-4dc36b52-9246-4b11-86c9-2195387b1c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:10.665411 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc36b52-9246-4b11-86c9-2195387b1c6e None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:10.671173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:10.671933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:10.672287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:10.672624 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] PATH_INFO: `/v3/projects/b0a9eed9af9a46d7a9c7512640bdd1b7` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:10.673926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0a9eed9af9a46d7a9c7512640bdd1b7)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:10.675983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:10.741075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0a9eed9af9a46d7a9c7512640bdd1b7', 'request_id': 'req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:10.748825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0a9eed9af9a46d7a9c7512640bdd1b7', 'request_id': 'req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:10.759521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0a9eed9af9a46d7a9c7512640bdd1b7', 'request_id': 'req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:10.773307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] The token cache is being invalidate because project b0a9eed9af9a46d7a9c7512640bdd1b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:56:10.773694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:10.775208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:10.776918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1156/2337] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:56:10 2022] DELETE /identity/v3/projects/b0a9eed9af9a46d7a9c7512640bdd1b7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:10.781725 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09038641-d07e-4dc0-98f7-6e01c89d152e service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:10.792309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09038641-d07e-4dc0-98f7-6e01c89d152e service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:10.854591 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5f181-a1d9-42e4-89cc-420d0be57556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:10.855537 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5f181-a1d9-42e4-89cc-420d0be57556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:10.855816 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5f181-a1d9-42e4-89cc-420d0be57556 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:10.856086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5f181-a1d9-42e4-89cc-420d0be57556 admin admin] PATH_INFO: `/v3/users/6d53b5590d9140c08768811044a34504` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:10.857272 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5f181-a1d9-42e4-89cc-420d0be57556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d53b5590d9140c08768811044a34504)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:10.859349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5f181-a1d9-42e4-89cc-420d0be57556 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:10.928767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-93b5f181-a1d9-42e4-89cc-420d0be57556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d53b5590d9140c08768811044a34504'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:10.943171 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-93b5f181-a1d9-42e4-89cc-420d0be57556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d53b5590d9140c08768811044a34504'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:10.959034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-93b5f181-a1d9-42e4-89cc-420d0be57556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d53b5590d9140c08768811044a34504'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:10.963270 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-93b5f181-a1d9-42e4-89cc-420d0be57556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d53b5590d9140c08768811044a34504'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:10.963706 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-93b5f181-a1d9-42e4-89cc-420d0be57556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d53b5590d9140c08768811044a34504'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:10.976052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:10.985453 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1182/2338] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:56:10 2022] DELETE /identity/v3/users/6d53b5590d9140c08768811044a34504 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:56:11.413909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:11.416890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:11.426822 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:11.433633 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae6f67-f00e-446d-b3eb-c46c04e0e1bf admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:11.507159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:11.508184 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:11.508488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:11.508741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88 admin admin] PATH_INFO: `/v3/projects/7b2fc6d0215c4e08b40f49f4b5248b63` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:11.509977 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b2fc6d0215c4e08b40f49f4b5248b63)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:11.512129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:11.565874 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b2fc6d0215c4e08b40f49f4b5248b63', 'request_id': 'req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:11.573576 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b2fc6d0215c4e08b40f49f4b5248b63', 'request_id': 'req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:11.583245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b2fc6d0215c4e08b40f49f4b5248b63', 'request_id': 'req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:11.597899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88 admin admin] The token cache is being invalidate because project 7b2fc6d0215c4e08b40f49f4b5248b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:56:11.598187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:11.598865 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2f9a5ce9-e741-4a8e-b4bf-91548d85dd88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:11.600825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1157/2339] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:56:11 2022] DELETE /identity/v3/projects/7b2fc6d0215c4e08b40f49f4b5248b63 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:56:11.606140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16786037-2248-4767-a444-458a2491a2d1 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:11.606419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16786037-2248-4767-a444-458a2491a2d1 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:11.606710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16786037-2248-4767-a444-458a2491a2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:11.628491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-16786037-2248-4767-a444-458a2491a2d1 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:11.629980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-16786037-2248-4767-a444-458a2491a2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:11.710469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1183/2340] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:11.718475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66e64e-4a8a-476b-b490-24679688270d None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:11.719138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66e64e-4a8a-476b-b490-24679688270d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:11.719370 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66e64e-4a8a-476b-b490-24679688270d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:11.744522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-9d66e64e-4a8a-476b-b490-24679688270d None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:11.745917 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9d66e64e-4a8a-476b-b490-24679688270d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:11.754274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1158/2341] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:11.759578 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16786037-2248-4767-a444-458a2491a2d1 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:11.770009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16786037-2248-4767-a444-458a2491a2d1 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:11.775614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb95f96-d36f-4561-bd0d-b60fb2311862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:11.776241 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb95f96-d36f-4561-bd0d-b60fb2311862 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:11.776518 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb95f96-d36f-4561-bd0d-b60fb2311862 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:11.776893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb95f96-d36f-4561-bd0d-b60fb2311862 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:11.777600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb95f96-d36f-4561-bd0d-b60fb2311862 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:56:11.777906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb95f96-d36f-4561-bd0d-b60fb2311862 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:11.779743 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb95f96-d36f-4561-bd0d-b60fb2311862 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:11.795449 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1184/2342] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:56:11.796519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66e64e-4a8a-476b-b490-24679688270d None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:11.806286 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66e64e-4a8a-476b-b490-24679688270d None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:11.811150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e824981-1c22-40cc-8ac4-f2a6cd2f05b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:11.811764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e824981-1c22-40cc-8ac4-f2a6cd2f05b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:11.812040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e824981-1c22-40cc-8ac4-f2a6cd2f05b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:11.812311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e824981-1c22-40cc-8ac4-f2a6cd2f05b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:11.812971 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e824981-1c22-40cc-8ac4-f2a6cd2f05b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:11.814958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e824981-1c22-40cc-8ac4-f2a6cd2f05b5 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:11.855304 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1159/2343] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:56:11 2022] POST /identity/v3/projects => generated 393 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:56:11.859947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb95f96-d36f-4561-bd0d-b60fb2311862 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:11.878950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb95f96-d36f-4561-bd0d-b60fb2311862 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:11.965057 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799c219-09fa-4ac0-94a4-6d5924b41f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:11.966100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799c219-09fa-4ac0-94a4-6d5924b41f81 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:11.966607 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799c219-09fa-4ac0-94a4-6d5924b41f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:11.967144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799c219-09fa-4ac0-94a4-6d5924b41f81 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:11.968369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799c219-09fa-4ac0-94a4-6d5924b41f81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:11.972207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799c219-09fa-4ac0-94a4-6d5924b41f81 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:12.012565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1185/2344] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:56:11 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:56:12.016614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e824981-1c22-40cc-8ac4-f2a6cd2f05b5 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:12.028424 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e824981-1c22-40cc-8ac4-f2a6cd2f05b5 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:12.036262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbf28c-25f1-4401-a81d-da3dc7d6b418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:12.036897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbf28c-25f1-4401-a81d-da3dc7d6b418 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:12.037234 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbf28c-25f1-4401-a81d-da3dc7d6b418 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:12.037497 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbf28c-25f1-4401-a81d-da3dc7d6b418 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:12.038210 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbf28c-25f1-4401-a81d-da3dc7d6b418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:12.039853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbf28c-25f1-4401-a81d-da3dc7d6b418 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:12.056696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1160/2345] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:12.058194 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799c219-09fa-4ac0-94a4-6d5924b41f81 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:12.078717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799c219-09fa-4ac0-94a4-6d5924b41f81 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:12.085500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76f467-4e2e-41c8-9f9a-d2674dfa73b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:12.095934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76f467-4e2e-41c8-9f9a-d2674dfa73b6 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:12.096203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76f467-4e2e-41c8-9f9a-d2674dfa73b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:12.096559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76f467-4e2e-41c8-9f9a-d2674dfa73b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:12.097298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76f467-4e2e-41c8-9f9a-d2674dfa73b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:12.098950 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76f467-4e2e-41c8-9f9a-d2674dfa73b6 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:12.110884 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1186/2346] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:12.116386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbf28c-25f1-4401-a81d-da3dc7d6b418 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:12.128238 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbf28c-25f1-4401-a81d-da3dc7d6b418 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:12.133783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1761d-43f2-4e13-a451-1d8610bbb72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:12.134455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1761d-43f2-4e13-a451-1d8610bbb72a admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:12.134764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1761d-43f2-4e13-a451-1d8610bbb72a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:12.135039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1761d-43f2-4e13-a451-1d8610bbb72a admin admin] PATH_INFO: `/v3/projects/901db208fb984593951fc1eca6050007/users/8fd6e7619c7c4fe2bd33b2c33c8dfb3f/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:12.151119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1761d-43f2-4e13-a451-1d8610bbb72a admin admin] RBAC: Authorizing `identity:create_grant(project_id=901db208fb984593951fc1eca6050007, user_id=8fd6e7619c7c4fe2bd33b2c33c8dfb3f, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:12.153885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1761d-43f2-4e13-a451-1d8610bbb72a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:12.179497 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1161/2347] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:56:12 2022] PUT /identity/v3/projects/901db208fb984593951fc1eca6050007/users/8fd6e7619c7c4fe2bd33b2c33c8dfb3f/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:12.182798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fbc69-2490-4e64-8490-62b5e79682d3 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:12.183247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fbc69-2490-4e64-8490-62b5e79682d3 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:12.183768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fbc69-2490-4e64-8490-62b5e79682d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:12.294033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-022fbc69-2490-4e64-8490-62b5e79682d3 None None] MFA Rules not processed for user `8fd6e7619c7c4fe2bd33b2c33c8dfb3f`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:12.296458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-022fbc69-2490-4e64-8490-62b5e79682d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:12.421742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:12.436322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1187/2348] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:12.440464 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1761d-43f2-4e13-a451-1d8610bbb72a admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:12.485892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:12.530603 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1761d-43f2-4e13-a451-1d8610bbb72a admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:12.536481 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26aded9-971b-42b5-b0d6-c02df225a98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:12.537303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26aded9-971b-42b5-b0d6-c02df225a98d admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:12.538112 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26aded9-971b-42b5-b0d6-c02df225a98d admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:12.538224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26aded9-971b-42b5-b0d6-c02df225a98d admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:12.539598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26aded9-971b-42b5-b0d6-c02df225a98d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:12.541630 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26aded9-971b-42b5-b0d6-c02df225a98d admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:12.581768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1162/2349] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:56:12 2022] POST /identity/v3/projects => generated 391 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:56:12.586754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022fbc69-2490-4e64-8490-62b5e79682d3 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:12.598171 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022fbc69-2490-4e64-8490-62b5e79682d3 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:12.675091 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2639d-ff1a-4235-9bbf-840441e5de6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:12.675775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2639d-ff1a-4235-9bbf-840441e5de6f admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:12.676044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2639d-ff1a-4235-9bbf-840441e5de6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:12.676325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2639d-ff1a-4235-9bbf-840441e5de6f admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:12.677005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2639d-ff1a-4235-9bbf-840441e5de6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:12.679382 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2639d-ff1a-4235-9bbf-840441e5de6f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:12.709059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1188/2350] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:56:12 2022] POST /identity/v3/users => generated 302 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:56:12.714912 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26aded9-971b-42b5-b0d6-c02df225a98d admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:12.726673 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26aded9-971b-42b5-b0d6-c02df225a98d admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:12.732211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8374bab-879c-4580-92db-d8169d118d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:12.732830 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8374bab-879c-4580-92db-d8169d118d6a admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:12.733109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8374bab-879c-4580-92db-d8169d118d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:12.733378 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8374bab-879c-4580-92db-d8169d118d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:12.734072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8374bab-879c-4580-92db-d8169d118d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:12.735699 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8374bab-879c-4580-92db-d8169d118d6a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:12.747612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1163/2351] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:12.752694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2639d-ff1a-4235-9bbf-840441e5de6f admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:12.765998 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2639d-ff1a-4235-9bbf-840441e5de6f admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:12.770907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2a48c-bc65-45a1-92ff-2d536ba9a127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:12.772108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2a48c-bc65-45a1-92ff-2d536ba9a127 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:12.772478 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2a48c-bc65-45a1-92ff-2d536ba9a127 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:12.772750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2a48c-bc65-45a1-92ff-2d536ba9a127 admin admin] PATH_INFO: `/v3/projects/4d3ac54f1d4a47ccaad37b64815dc317/users/f050b98e5ef9422792bf18decc045017/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:12.789457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2a48c-bc65-45a1-92ff-2d536ba9a127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d3ac54f1d4a47ccaad37b64815dc317, user_id=f050b98e5ef9422792bf18decc045017, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:12.792225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2a48c-bc65-45a1-92ff-2d536ba9a127 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:12.806947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1189/2352] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:56:12 2022] PUT /identity/v3/projects/4d3ac54f1d4a47ccaad37b64815dc317/users/f050b98e5ef9422792bf18decc045017/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:12.811242 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8374bab-879c-4580-92db-d8169d118d6a admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:12.824480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8374bab-879c-4580-92db-d8169d118d6a admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:12.896259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abb0dc-6b85-4e7e-bbda-3088f98053c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:12.897638 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abb0dc-6b85-4e7e-bbda-3088f98053c1 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:12.897922 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abb0dc-6b85-4e7e-bbda-3088f98053c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:12.898275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abb0dc-6b85-4e7e-bbda-3088f98053c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:12.899004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abb0dc-6b85-4e7e-bbda-3088f98053c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:12.900742 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abb0dc-6b85-4e7e-bbda-3088f98053c1 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:12.911989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1164/2353] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:12.917354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2a48c-bc65-45a1-92ff-2d536ba9a127 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:12.928904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2a48c-bc65-45a1-92ff-2d536ba9a127 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:12.933510 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e28f7-d49d-4325-8e46-1136249c0eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:12.934176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e28f7-d49d-4325-8e46-1136249c0eee admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:12.934454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e28f7-d49d-4325-8e46-1136249c0eee admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:12.934771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e28f7-d49d-4325-8e46-1136249c0eee admin admin] PATH_INFO: `/v3/projects/4d3ac54f1d4a47ccaad37b64815dc317/users/f050b98e5ef9422792bf18decc045017/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:12.936695 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e28f7-d49d-4325-8e46-1136249c0eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d3ac54f1d4a47ccaad37b64815dc317, user_id=f050b98e5ef9422792bf18decc045017, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:12.940098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e28f7-d49d-4325-8e46-1136249c0eee admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:12.954834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1190/2354] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:56:12 2022] PUT /identity/v3/projects/4d3ac54f1d4a47ccaad37b64815dc317/users/f050b98e5ef9422792bf18decc045017/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:12.967176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca76c9-c7bd-4458-8a3b-ba7e3bd03c66 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:12.967469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca76c9-c7bd-4458-8a3b-ba7e3bd03c66 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:12.967746 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca76c9-c7bd-4458-8a3b-ba7e3bd03c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:12.997628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-24ca76c9-c7bd-4458-8a3b-ba7e3bd03c66 None None] MFA Rules not processed for user `f050b98e5ef9422792bf18decc045017`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:12.998949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-24ca76c9-c7bd-4458-8a3b-ba7e3bd03c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:13.077706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1165/2355] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:12 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:56:13.089383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e28f7-d49d-4325-8e46-1136249c0eee admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:13.091198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9a2e28f7-d49d-4325-8e46-1136249c0eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:13.154317 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e28f7-d49d-4325-8e46-1136249c0eee admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:13.157942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ffabc-6ee9-46ec-93e8-a89401f10a85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:13.158554 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ffabc-6ee9-46ec-93e8-a89401f10a85 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:13.158921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ffabc-6ee9-46ec-93e8-a89401f10a85 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:13.159193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ffabc-6ee9-46ec-93e8-a89401f10a85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:13.170255 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ffabc-6ee9-46ec-93e8-a89401f10a85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:13.171743 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ffabc-6ee9-46ec-93e8-a89401f10a85 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:13.184537 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1191/2356] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:13 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:13.241996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca76c9-c7bd-4458-8a3b-ba7e3bd03c66 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:13.243434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-24ca76c9-c7bd-4458-8a3b-ba7e3bd03c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:13.256451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca76c9-c7bd-4458-8a3b-ba7e3bd03c66 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:13.260300 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee4975-a0a8-473e-b515-c41a1114b606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:13.260922 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee4975-a0a8-473e-b515-c41a1114b606 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:13.261208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee4975-a0a8-473e-b515-c41a1114b606 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:13.261476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee4975-a0a8-473e-b515-c41a1114b606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:13.283812 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:13.323857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee4975-a0a8-473e-b515-c41a1114b606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:13.325290 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee4975-a0a8-473e-b515-c41a1114b606 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:13.335686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1166/2357] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:56:16.914601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ffabc-6ee9-46ec-93e8-a89401f10a85 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:16.973412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:16.988325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ffabc-6ee9-46ec-93e8-a89401f10a85 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:16.994690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:16.995342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:16.995665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:16.996055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] PATH_INFO: `/v3/users/8fd6e7619c7c4fe2bd33b2c33c8dfb3f` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:16.998413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd6e7619c7c4fe2bd33b2c33c8dfb3f)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:17.000697 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:17.061230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd6e7619c7c4fe2bd33b2c33c8dfb3f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:17.083231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd6e7619c7c4fe2bd33b2c33c8dfb3f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:17.095844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd6e7619c7c4fe2bd33b2c33c8dfb3f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:17.096607 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd6e7619c7c4fe2bd33b2c33c8dfb3f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:17.097255 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd6e7619c7c4fe2bd33b2c33c8dfb3f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:17.113640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1192/2358] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:56:16 2022] DELETE /identity/v3/users/8fd6e7619c7c4fe2bd33b2c33c8dfb3f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:56:17.120406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353712c5-9eaa-4fd7-b029-3d982075cb7b None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:17.121504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353712c5-9eaa-4fd7-b029-3d982075cb7b None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:17.122211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353712c5-9eaa-4fd7-b029-3d982075cb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:17.147668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-353712c5-9eaa-4fd7-b029-3d982075cb7b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:17.148963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-353712c5-9eaa-4fd7-b029-3d982075cb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:17.235714 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1167/2359] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:17.247255 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:17.248934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:17.317818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3027c9-6398-4347-af1e-eb757b83a679 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:17.322974 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d6bb1-8877-4306-99e3-5861fd109b89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:17.323693 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d6bb1-8877-4306-99e3-5861fd109b89 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:17.323979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d6bb1-8877-4306-99e3-5861fd109b89 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:17.324298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d6bb1-8877-4306-99e3-5861fd109b89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:17.337117 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d6bb1-8877-4306-99e3-5861fd109b89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:17.339555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d6bb1-8877-4306-99e3-5861fd109b89 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:17.353269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1193/2360] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:17.830741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353712c5-9eaa-4fd7-b029-3d982075cb7b None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:17.832701 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-353712c5-9eaa-4fd7-b029-3d982075cb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:17.853140 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353712c5-9eaa-4fd7-b029-3d982075cb7b None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:17.860091 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:17.861078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:17.861946 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:17.862518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] PATH_INFO: `/v3/projects/901db208fb984593951fc1eca6050007` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:17.864527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=901db208fb984593951fc1eca6050007)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:17.867758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:17.934022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901db208fb984593951fc1eca6050007', 'request_id': 'req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:17.940685 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901db208fb984593951fc1eca6050007', 'request_id': 'req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:17.951704 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901db208fb984593951fc1eca6050007', 'request_id': 'req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:17.968247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] The token cache is being invalidate because project 901db208fb984593951fc1eca6050007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:56:17.968779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:17.969854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:17.972418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1168/2361] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:56:17 2022] DELETE /identity/v3/projects/901db208fb984593951fc1eca6050007 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:17.977158 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d6bb1-8877-4306-99e3-5861fd109b89 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:17.989134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d6bb1-8877-4306-99e3-5861fd109b89 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:18.093793 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:18.094243 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:18.095456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:18.095456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] PATH_INFO: `/v3/users/f050b98e5ef9422792bf18decc045017` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:18.097664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=f050b98e5ef9422792bf18decc045017)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:18.101291 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:18.129563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:18.139151 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:18.140940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:18.184602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f050b98e5ef9422792bf18decc045017'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:18.199545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f050b98e5ef9422792bf18decc045017'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:18.212994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f050b98e5ef9422792bf18decc045017'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:18.214330 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f050b98e5ef9422792bf18decc045017'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:18.215137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f050b98e5ef9422792bf18decc045017'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:18.237978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1194/2362] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:56:17 2022] DELETE /identity/v3/users/f050b98e5ef9422792bf18decc045017 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:56:18.254227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:18.262091 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc1d01-60b8-47d4-8d52-afe593c73d1c admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:18.342894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909433e-1db0-48ea-bd5c-d11b0a289cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:18.343921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909433e-1db0-48ea-bd5c-d11b0a289cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:18.344433 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909433e-1db0-48ea-bd5c-d11b0a289cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:18.345471 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909433e-1db0-48ea-bd5c-d11b0a289cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:18.349451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3909433e-1db0-48ea-bd5c-d11b0a289cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:18.457108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909433e-1db0-48ea-bd5c-d11b0a289cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:18.458826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909433e-1db0-48ea-bd5c-d11b0a289cb8 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:18.470148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1169/2363] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:56:18.712921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:18.714468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:18.836418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20670c7-9e2a-4a8b-8bcf-ab13129066ca admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:18.847427 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18667e-dd67-4c05-9587-579f9ee775cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:18.848102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18667e-dd67-4c05-9587-579f9ee775cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:18.848386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18667e-dd67-4c05-9587-579f9ee775cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:18.848663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18667e-dd67-4c05-9587-579f9ee775cc admin admin] PATH_INFO: `/v3/projects/4d3ac54f1d4a47ccaad37b64815dc317` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:18.850466 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18667e-dd67-4c05-9587-579f9ee775cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d3ac54f1d4a47ccaad37b64815dc317)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:18.852503 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18667e-dd67-4c05-9587-579f9ee775cc admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:18.919580 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c18667e-dd67-4c05-9587-579f9ee775cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d3ac54f1d4a47ccaad37b64815dc317', 'request_id': 'req-1c18667e-dd67-4c05-9587-579f9ee775cc'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:18.926747 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c18667e-dd67-4c05-9587-579f9ee775cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d3ac54f1d4a47ccaad37b64815dc317', 'request_id': 'req-1c18667e-dd67-4c05-9587-579f9ee775cc'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:18.937684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c18667e-dd67-4c05-9587-579f9ee775cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d3ac54f1d4a47ccaad37b64815dc317', 'request_id': 'req-1c18667e-dd67-4c05-9587-579f9ee775cc'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:18.950599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c18667e-dd67-4c05-9587-579f9ee775cc admin admin] The token cache is being invalidate because project 4d3ac54f1d4a47ccaad37b64815dc317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:56:18.951067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c18667e-dd67-4c05-9587-579f9ee775cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:18.951730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c18667e-dd67-4c05-9587-579f9ee775cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:18.953342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1195/2364] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:56:18 2022] DELETE /identity/v3/projects/4d3ac54f1d4a47ccaad37b64815dc317 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:18.960295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0941c-b72d-4965-b833-96f90ac9a451 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:18.960598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0941c-b72d-4965-b833-96f90ac9a451 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:18.960867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0941c-b72d-4965-b833-96f90ac9a451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:18.984742 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-84f0941c-b72d-4965-b833-96f90ac9a451 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:18.985963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-84f0941c-b72d-4965-b833-96f90ac9a451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:19.066734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1170/2365] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:19.076111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc580ccd-840b-43d1-ae0a-fe119fada7db None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:19.076626 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc580ccd-840b-43d1-ae0a-fe119fada7db None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:19.076745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc580ccd-840b-43d1-ae0a-fe119fada7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:19.096845 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-fc580ccd-840b-43d1-ae0a-fe119fada7db None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:19.098047 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fc580ccd-840b-43d1-ae0a-fe119fada7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:19.107504 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1196/2366] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:19.111940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0941c-b72d-4965-b833-96f90ac9a451 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:19.124370 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0941c-b72d-4965-b833-96f90ac9a451 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:19.129171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733597a-eb4b-4f2f-b339-c827a604cdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:19.129800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733597a-eb4b-4f2f-b339-c827a604cdc7 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:19.130074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733597a-eb4b-4f2f-b339-c827a604cdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:19.130343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733597a-eb4b-4f2f-b339-c827a604cdc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:19.132449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733597a-eb4b-4f2f-b339-c827a604cdc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:56:19.132760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733597a-eb4b-4f2f-b339-c827a604cdc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:19.134411 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733597a-eb4b-4f2f-b339-c827a604cdc7 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:19.147544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1171/2367] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:56:19.152497 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc580ccd-840b-43d1-ae0a-fe119fada7db None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:19.163824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc580ccd-840b-43d1-ae0a-fe119fada7db None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:19.183274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a45fd8-ada5-4df8-8008-888df6d2dbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:19.183916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a45fd8-ada5-4df8-8008-888df6d2dbf6 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:19.184195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a45fd8-ada5-4df8-8008-888df6d2dbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:19.184465 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a45fd8-ada5-4df8-8008-888df6d2dbf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:19.185139 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a45fd8-ada5-4df8-8008-888df6d2dbf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:19.187099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a45fd8-ada5-4df8-8008-888df6d2dbf6 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:19.207823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:19.227945 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1197/2368] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:56:19 2022] POST /identity/v3/projects => generated 367 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:56:19.232230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733597a-eb4b-4f2f-b339-c827a604cdc7 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:19.244770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733597a-eb4b-4f2f-b339-c827a604cdc7 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:19.291946 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:19.311595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef04b7-efb6-4889-b9ab-7b5bfa89d840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:19.312486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef04b7-efb6-4889-b9ab-7b5bfa89d840 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:19.312774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef04b7-efb6-4889-b9ab-7b5bfa89d840 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:19.313049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef04b7-efb6-4889-b9ab-7b5bfa89d840 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:19.313753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef04b7-efb6-4889-b9ab-7b5bfa89d840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:19.315813 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef04b7-efb6-4889-b9ab-7b5bfa89d840 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:19.342468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1172/2369] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:56:19 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:56:19.346700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a45fd8-ada5-4df8-8008-888df6d2dbf6 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:19.358191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a45fd8-ada5-4df8-8008-888df6d2dbf6 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:19.362955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650edb28-c20b-4e52-8662-2f952a0790f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:19.363776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650edb28-c20b-4e52-8662-2f952a0790f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:19.364066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650edb28-c20b-4e52-8662-2f952a0790f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:19.364331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650edb28-c20b-4e52-8662-2f952a0790f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:19.365628 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650edb28-c20b-4e52-8662-2f952a0790f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:19.367442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650edb28-c20b-4e52-8662-2f952a0790f8 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:19.378993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1198/2370] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:56:19.383783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef04b7-efb6-4889-b9ab-7b5bfa89d840 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:19.395719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef04b7-efb6-4889-b9ab-7b5bfa89d840 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:19.401300 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8a3a0-3652-4d81-9bbd-6c10bb4d901e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:19.401953 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8a3a0-3652-4d81-9bbd-6c10bb4d901e admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:19.402233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8a3a0-3652-4d81-9bbd-6c10bb4d901e admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:19.402780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8a3a0-3652-4d81-9bbd-6c10bb4d901e admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:19.403487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8a3a0-3652-4d81-9bbd-6c10bb4d901e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:19.405091 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8a3a0-3652-4d81-9bbd-6c10bb4d901e admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:19.418533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1173/2371] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:19.422880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650edb28-c20b-4e52-8662-2f952a0790f8 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:19.433826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650edb28-c20b-4e52-8662-2f952a0790f8 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:19.438419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a204e3b-5d23-48f7-8304-6176a5cf9203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:19.439117 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a204e3b-5d23-48f7-8304-6176a5cf9203 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:19.439396 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a204e3b-5d23-48f7-8304-6176a5cf9203 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:19.439663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a204e3b-5d23-48f7-8304-6176a5cf9203 admin admin] PATH_INFO: `/v3/projects/44cde67e8bd043e0a7d34b366e382b7f/users/6df31c5bf3784bbdbd640e9d05b0be49/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:19.455234 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a204e3b-5d23-48f7-8304-6176a5cf9203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44cde67e8bd043e0a7d34b366e382b7f, user_id=6df31c5bf3784bbdbd640e9d05b0be49, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:19.457969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a204e3b-5d23-48f7-8304-6176a5cf9203 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:19.473989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1199/2372] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:56:19 2022] PUT /identity/v3/projects/44cde67e8bd043e0a7d34b366e382b7f/users/6df31c5bf3784bbdbd640e9d05b0be49/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:19.490053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfdc87-19a9-4fd5-8752-25df3bb67fb1 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:19.490504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfdc87-19a9-4fd5-8752-25df3bb67fb1 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:19.491046 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfdc87-19a9-4fd5-8752-25df3bb67fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:19.540210 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-f9cfdc87-19a9-4fd5-8752-25df3bb67fb1 None None] MFA Rules not processed for user `6df31c5bf3784bbdbd640e9d05b0be49`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:19.542274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f9cfdc87-19a9-4fd5-8752-25df3bb67fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:19.670520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1174/2373] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:19.673574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a204e3b-5d23-48f7-8304-6176a5cf9203 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:19.747650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a204e3b-5d23-48f7-8304-6176a5cf9203 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:19.752903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dc8e6-65cf-4006-b426-8a8a1d116fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:19.753658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dc8e6-65cf-4006-b426-8a8a1d116fcd admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:19.753780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dc8e6-65cf-4006-b426-8a8a1d116fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:19.754050 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dc8e6-65cf-4006-b426-8a8a1d116fcd admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:19.754740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dc8e6-65cf-4006-b426-8a8a1d116fcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:19.757746 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dc8e6-65cf-4006-b426-8a8a1d116fcd admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:19.791677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1200/2374] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:56:19 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:56:19.797077 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfdc87-19a9-4fd5-8752-25df3bb67fb1 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:19.817322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfdc87-19a9-4fd5-8752-25df3bb67fb1 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:19.922731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0ff59-dc88-45f7-a88d-7e6644bf3505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:19.923472 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0ff59-dc88-45f7-a88d-7e6644bf3505 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:19.923884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0ff59-dc88-45f7-a88d-7e6644bf3505 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:19.924254 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0ff59-dc88-45f7-a88d-7e6644bf3505 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:19.925362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0ff59-dc88-45f7-a88d-7e6644bf3505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:19.928696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0ff59-dc88-45f7-a88d-7e6644bf3505 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:19.958870 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1175/2375] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:56:19 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:56:19.961512 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dc8e6-65cf-4006-b426-8a8a1d116fcd admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:19.975132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dc8e6-65cf-4006-b426-8a8a1d116fcd admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:19.982996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484a7a4-8a52-4538-afe5-9ad6c1dd88e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:19.983277 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484a7a4-8a52-4538-afe5-9ad6c1dd88e1 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:19.983674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484a7a4-8a52-4538-afe5-9ad6c1dd88e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:19.984008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484a7a4-8a52-4538-afe5-9ad6c1dd88e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:19.985187 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484a7a4-8a52-4538-afe5-9ad6c1dd88e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:19.987065 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484a7a4-8a52-4538-afe5-9ad6c1dd88e1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:20.000355 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1201/2376] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:20.005186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0ff59-dc88-45f7-a88d-7e6644bf3505 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:20.020317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0ff59-dc88-45f7-a88d-7e6644bf3505 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:20.030226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44963a-a668-4122-9a00-d9dbca34a096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:20.031716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44963a-a668-4122-9a00-d9dbca34a096 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:20.032032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44963a-a668-4122-9a00-d9dbca34a096 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:20.032295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44963a-a668-4122-9a00-d9dbca34a096 admin admin] PATH_INFO: `/v3/projects/6ffdd71ce8c04aada41edb72c78a8517/users/847b411100dc419da157d5218089df1c/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:20.072093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44963a-a668-4122-9a00-d9dbca34a096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffdd71ce8c04aada41edb72c78a8517, user_id=847b411100dc419da157d5218089df1c, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:20.079227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44963a-a668-4122-9a00-d9dbca34a096 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:20.109876 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1176/2377] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:56:20 2022] PUT /identity/v3/projects/6ffdd71ce8c04aada41edb72c78a8517/users/847b411100dc419da157d5218089df1c/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:20.114211 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484a7a4-8a52-4538-afe5-9ad6c1dd88e1 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:20.129640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484a7a4-8a52-4538-afe5-9ad6c1dd88e1 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:20.216942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3effd-18a4-4898-a47e-df5a54af8f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:20.217893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3effd-18a4-4898-a47e-df5a54af8f5e admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:20.218173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3effd-18a4-4898-a47e-df5a54af8f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:20.218449 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3effd-18a4-4898-a47e-df5a54af8f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:20.219749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3effd-18a4-4898-a47e-df5a54af8f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:20.222173 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3effd-18a4-4898-a47e-df5a54af8f5e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:20.237750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1202/2378] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:20.241115 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44963a-a668-4122-9a00-d9dbca34a096 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:20.266690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44963a-a668-4122-9a00-d9dbca34a096 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:20.281357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50f13b-8805-461c-b2ad-dd7c398c66e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:20.282843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50f13b-8805-461c-b2ad-dd7c398c66e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:20.283956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50f13b-8805-461c-b2ad-dd7c398c66e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:20.284354 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50f13b-8805-461c-b2ad-dd7c398c66e4 admin admin] PATH_INFO: `/v3/projects/6ffdd71ce8c04aada41edb72c78a8517/users/847b411100dc419da157d5218089df1c/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:20.287039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50f13b-8805-461c-b2ad-dd7c398c66e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffdd71ce8c04aada41edb72c78a8517, user_id=847b411100dc419da157d5218089df1c, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:20.290411 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50f13b-8805-461c-b2ad-dd7c398c66e4 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:20.304171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:20.314946 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1177/2379] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:56:20 2022] PUT /identity/v3/projects/6ffdd71ce8c04aada41edb72c78a8517/users/847b411100dc419da157d5218089df1c/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:20.328996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e179d81-7942-4c12-81d6-39ed0869cedf None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:20.329594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e179d81-7942-4c12-81d6-39ed0869cedf None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:20.330039 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e179d81-7942-4c12-81d6-39ed0869cedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:20.373259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3e179d81-7942-4c12-81d6-39ed0869cedf None None] MFA Rules not processed for user `847b411100dc419da157d5218089df1c`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:20.374675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3e179d81-7942-4c12-81d6-39ed0869cedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:20.398645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:20.541821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1203/2380] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:20.556876 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50f13b-8805-461c-b2ad-dd7c398c66e4 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:20.558486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-1f50f13b-8805-461c-b2ad-dd7c398c66e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:20.571006 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ab8ec-0694-4241-954e-32f2f90a5e97 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:20.571301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ab8ec-0694-4241-954e-32f2f90a5e97 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:20.571664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ab8ec-0694-4241-954e-32f2f90a5e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:20.609034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3a6ab8ec-0694-4241-954e-32f2f90a5e97 None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:20.611023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3a6ab8ec-0694-4241-954e-32f2f90a5e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:20.640854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50f13b-8805-461c-b2ad-dd7c398c66e4 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:20.646761 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5cbd5-1460-4448-bd48-5cce0d0898db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:20.647748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5cbd5-1460-4448-bd48-5cce0d0898db service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:20.648188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5cbd5-1460-4448-bd48-5cce0d0898db service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:20.648616 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5cbd5-1460-4448-bd48-5cce0d0898db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:20.664131 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5cbd5-1460-4448-bd48-5cce0d0898db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:20.666536 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5cbd5-1460-4448-bd48-5cce0d0898db service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:20.681579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1178/2381] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:20 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:56:20.696233 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1204/2382] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:56:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:56:20.712888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5cbd5-1460-4448-bd48-5cce0d0898db service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:20.716169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f7d5cbd5-1460-4448-bd48-5cce0d0898db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:20.799829 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ab8ec-0694-4241-954e-32f2f90a5e97 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:20.811512 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ab8ec-0694-4241-954e-32f2f90a5e97 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:20.816769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf68948-1c4e-47fb-8a8c-167b9adc17ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:20.817680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf68948-1c4e-47fb-8a8c-167b9adc17ed service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:20.818476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf68948-1c4e-47fb-8a8c-167b9adc17ed service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:20.819206 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf68948-1c4e-47fb-8a8c-167b9adc17ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:20.832108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5cbd5-1460-4448-bd48-5cce0d0898db service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:20.837705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e2db1-c21f-41db-b614-137645a5cd6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:20.838322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e2db1-c21f-41db-b614-137645a5cd6a service swift] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:20.838679 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e2db1-c21f-41db-b614-137645a5cd6a service swift] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:20.839055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e2db1-c21f-41db-b614-137645a5cd6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:20.848388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e2db1-c21f-41db-b614-137645a5cd6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:20.850554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e2db1-c21f-41db-b614-137645a5cd6a service swift] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:20.860394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1179/2383] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:56:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 19:56:20.911417 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf68948-1c4e-47fb-8a8c-167b9adc17ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:20.916439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf68948-1c4e-47fb-8a8c-167b9adc17ed service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:20.933683 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1205/2384] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:56:28.069019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e2db1-c21f-41db-b614-137645a5cd6a service swift] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:28.174371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e2db1-c21f-41db-b614-137645a5cd6a service swift] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:28.180557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:28.181371 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:28.181710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:28.182047 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] PATH_INFO: `/v3/users/6df31c5bf3784bbdbd640e9d05b0be49` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:28.183307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6df31c5bf3784bbdbd640e9d05b0be49)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:28.185299 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:28.238430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:28.250480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6df31c5bf3784bbdbd640e9d05b0be49'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:28.259231 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6df31c5bf3784bbdbd640e9d05b0be49'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:28.260370 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6df31c5bf3784bbdbd640e9d05b0be49'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:28.274924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6df31c5bf3784bbdbd640e9d05b0be49'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:28.283897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6df31c5bf3784bbdbd640e9d05b0be49'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:28.292268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1180/2385] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:56:28 2022] DELETE /identity/v3/users/6df31c5bf3784bbdbd640e9d05b0be49 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:28.298389 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c221a0-a0dd-4d88-a149-070f64554d4e None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:28.299135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c221a0-a0dd-4d88-a149-070f64554d4e None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:28.299135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c221a0-a0dd-4d88-a149-070f64554d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:28.335655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-30c221a0-a0dd-4d88-a149-070f64554d4e None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:28.336924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-30c221a0-a0dd-4d88-a149-070f64554d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:28.405921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:28.418648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1206/2386] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:28.426589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:28.428780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:28.514911 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030da04-a724-46d6-ae9b-b7f83bfae1b7 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:28.521606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f8e8d-6d7e-4e2e-81ea-ee11a182a493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:28.523614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f8e8d-6d7e-4e2e-81ea-ee11a182a493 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:28.523614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f8e8d-6d7e-4e2e-81ea-ee11a182a493 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:28.523614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f8e8d-6d7e-4e2e-81ea-ee11a182a493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:28.533896 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f8e8d-6d7e-4e2e-81ea-ee11a182a493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:28.535390 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f8e8d-6d7e-4e2e-81ea-ee11a182a493 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:28.545994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1181/2387] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:29.016773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c221a0-a0dd-4d88-a149-070f64554d4e None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:29.019608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-30c221a0-a0dd-4d88-a149-070f64554d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:29.066434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c221a0-a0dd-4d88-a149-070f64554d4e None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:29.087730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:29.088445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:29.088921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:29.089397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] PATH_INFO: `/v3/projects/44cde67e8bd043e0a7d34b366e382b7f` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:29.091343 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44cde67e8bd043e0a7d34b366e382b7f)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:29.093593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:29.198319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44cde67e8bd043e0a7d34b366e382b7f', 'request_id': 'req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:29.221804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44cde67e8bd043e0a7d34b366e382b7f', 'request_id': 'req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:29.243685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44cde67e8bd043e0a7d34b366e382b7f', 'request_id': 'req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:29.264523 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] The token cache is being invalidate because project 44cde67e8bd043e0a7d34b366e382b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:56:29.266086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:29.282579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:29.282579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1207/2388] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:56:29 2022] DELETE /identity/v3/projects/44cde67e8bd043e0a7d34b366e382b7f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:29.287959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f8e8d-6d7e-4e2e-81ea-ee11a182a493 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:29.298508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f8e8d-6d7e-4e2e-81ea-ee11a182a493 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:29.403609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50904d-ed99-411a-be7f-376db12d5ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:29.405046 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50904d-ed99-411a-be7f-376db12d5ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:29.405919 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50904d-ed99-411a-be7f-376db12d5ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:29.406879 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50904d-ed99-411a-be7f-376db12d5ab8 admin admin] PATH_INFO: `/v3/users/847b411100dc419da157d5218089df1c` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:29.409293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50904d-ed99-411a-be7f-376db12d5ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=847b411100dc419da157d5218089df1c)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:29.417800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50904d-ed99-411a-be7f-376db12d5ab8 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:29.495839 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2f50904d-ed99-411a-be7f-376db12d5ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '847b411100dc419da157d5218089df1c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:29.509153 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2f50904d-ed99-411a-be7f-376db12d5ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '847b411100dc419da157d5218089df1c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:29.510697 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2f50904d-ed99-411a-be7f-376db12d5ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '847b411100dc419da157d5218089df1c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:29.516367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:29.532332 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2f50904d-ed99-411a-be7f-376db12d5ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '847b411100dc419da157d5218089df1c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:29.544232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2f50904d-ed99-411a-be7f-376db12d5ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '847b411100dc419da157d5218089df1c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:29.551893 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1182/2389] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:56:29 2022] DELETE /identity/v3/users/847b411100dc419da157d5218089df1c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:30.105662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:30.108566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:30.131277 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aecc3c-09fc-4b7c-a7bb-0b39ecb1a051 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:30.250376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293b9fd-baba-4372-bc72-44a130ce45f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:30.252947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293b9fd-baba-4372-bc72-44a130ce45f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:30.253722 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293b9fd-baba-4372-bc72-44a130ce45f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:30.254178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293b9fd-baba-4372-bc72-44a130ce45f7 admin admin] PATH_INFO: `/v3/projects/6ffdd71ce8c04aada41edb72c78a8517` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:30.255773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293b9fd-baba-4372-bc72-44a130ce45f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ffdd71ce8c04aada41edb72c78a8517)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:30.257878 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293b9fd-baba-4372-bc72-44a130ce45f7 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:30.374616 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c293b9fd-baba-4372-bc72-44a130ce45f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ffdd71ce8c04aada41edb72c78a8517', 'request_id': 'req-c293b9fd-baba-4372-bc72-44a130ce45f7'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:30.382956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c293b9fd-baba-4372-bc72-44a130ce45f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ffdd71ce8c04aada41edb72c78a8517', 'request_id': 'req-c293b9fd-baba-4372-bc72-44a130ce45f7'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:30.398276 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c293b9fd-baba-4372-bc72-44a130ce45f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ffdd71ce8c04aada41edb72c78a8517', 'request_id': 'req-c293b9fd-baba-4372-bc72-44a130ce45f7'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:30.416132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c293b9fd-baba-4372-bc72-44a130ce45f7 admin admin] The token cache is being invalidate because project 6ffdd71ce8c04aada41edb72c78a8517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:56:30.416510 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c293b9fd-baba-4372-bc72-44a130ce45f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:30.417279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c293b9fd-baba-4372-bc72-44a130ce45f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:30.418934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1208/2390] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:56:30 2022] DELETE /identity/v3/projects/6ffdd71ce8c04aada41edb72c78a8517 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:30.426809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca1d40-5c3a-4b88-9ed5-5dad3f18da07 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:30.427694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca1d40-5c3a-4b88-9ed5-5dad3f18da07 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:30.428040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca1d40-5c3a-4b88-9ed5-5dad3f18da07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:30.465719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-0eca1d40-5c3a-4b88-9ed5-5dad3f18da07 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:30.467744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-0eca1d40-5c3a-4b88-9ed5-5dad3f18da07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:30.624263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1183/2391] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:30.635967 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad6e75-58bd-410a-b8c4-d53990e2645c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:30.636463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad6e75-58bd-410a-b8c4-d53990e2645c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:30.637175 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad6e75-58bd-410a-b8c4-d53990e2645c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:30.663136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-08ad6e75-58bd-410a-b8c4-d53990e2645c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:30.665056 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-08ad6e75-58bd-410a-b8c4-d53990e2645c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:30.674717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1209/2392] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:30.681711 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca1d40-5c3a-4b88-9ed5-5dad3f18da07 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:30.694939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca1d40-5c3a-4b88-9ed5-5dad3f18da07 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:30.716599 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852dd9ca-798d-46a4-9caf-669658bda5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:30.717636 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852dd9ca-798d-46a4-9caf-669658bda5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:30.718020 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852dd9ca-798d-46a4-9caf-669658bda5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:30.719175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852dd9ca-798d-46a4-9caf-669658bda5e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:30.721135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852dd9ca-798d-46a4-9caf-669658bda5e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:56:30.721135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852dd9ca-798d-46a4-9caf-669658bda5e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:30.723255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852dd9ca-798d-46a4-9caf-669658bda5e2 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:30.752951 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1184/2393] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:56:30.754754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad6e75-58bd-410a-b8c4-d53990e2645c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:30.768071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:30.776556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad6e75-58bd-410a-b8c4-d53990e2645c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:30.793093 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba151f-b605-46a9-adc0-cfe8cb832286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:30.794510 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba151f-b605-46a9-adc0-cfe8cb832286 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:30.795419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba151f-b605-46a9-adc0-cfe8cb832286 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:30.795419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba151f-b605-46a9-adc0-cfe8cb832286 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:30.796030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba151f-b605-46a9-adc0-cfe8cb832286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:30.799462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba151f-b605-46a9-adc0-cfe8cb832286 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:30.848120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1210/2394] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:56:30 2022] POST /identity/v3/projects => generated 375 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:56:30.854788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852dd9ca-798d-46a4-9caf-669658bda5e2 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:30.869681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852dd9ca-798d-46a4-9caf-669658bda5e2 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:30.948520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae23376-cd4b-4b0f-aadb-f5641ae46a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:30.949209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae23376-cd4b-4b0f-aadb-f5641ae46a8d admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:30.949492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae23376-cd4b-4b0f-aadb-f5641ae46a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:30.949768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae23376-cd4b-4b0f-aadb-f5641ae46a8d admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:30.950435 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae23376-cd4b-4b0f-aadb-f5641ae46a8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:30.952487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae23376-cd4b-4b0f-aadb-f5641ae46a8d admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:30.981434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1185/2395] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:56:30 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:56:30.986971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba151f-b605-46a9-adc0-cfe8cb832286 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:31.009019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba151f-b605-46a9-adc0-cfe8cb832286 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:31.016820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22907da0-4c12-4e13-8e37-eae18dd03565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:31.017451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22907da0-4c12-4e13-8e37-eae18dd03565 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:31.017727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22907da0-4c12-4e13-8e37-eae18dd03565 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:31.017993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22907da0-4c12-4e13-8e37-eae18dd03565 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:31.018985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22907da0-4c12-4e13-8e37-eae18dd03565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:31.020458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22907da0-4c12-4e13-8e37-eae18dd03565 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:31.039363 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1211/2396] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:31.041124 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae23376-cd4b-4b0f-aadb-f5641ae46a8d admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:31.055871 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae23376-cd4b-4b0f-aadb-f5641ae46a8d admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:31.061243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db8d07-5929-40c9-9c60-97450945507c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:31.062580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db8d07-5929-40c9-9c60-97450945507c admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:31.062580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db8d07-5929-40c9-9c60-97450945507c admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:31.062580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db8d07-5929-40c9-9c60-97450945507c admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:31.063209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db8d07-5929-40c9-9c60-97450945507c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:31.065832 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db8d07-5929-40c9-9c60-97450945507c admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:31.087433 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1186/2397] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:56:31.088659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22907da0-4c12-4e13-8e37-eae18dd03565 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:31.102699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22907da0-4c12-4e13-8e37-eae18dd03565 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:31.107906 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc2877-83b0-4028-a546-a7b420a38966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:31.108795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc2877-83b0-4028-a546-a7b420a38966 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:31.109189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc2877-83b0-4028-a546-a7b420a38966 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:31.109562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc2877-83b0-4028-a546-a7b420a38966 admin admin] PATH_INFO: `/v3/projects/9a5122857f764dcf977389f772000b4e/users/fb77bfeec5c14d6db3f7a5eb71a6d79f/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:31.127881 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc2877-83b0-4028-a546-a7b420a38966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a5122857f764dcf977389f772000b4e, user_id=fb77bfeec5c14d6db3f7a5eb71a6d79f, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:31.130868 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc2877-83b0-4028-a546-a7b420a38966 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:31.157584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1212/2398] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:56:31 2022] PUT /identity/v3/projects/9a5122857f764dcf977389f772000b4e/users/fb77bfeec5c14d6db3f7a5eb71a6d79f/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:31.169823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10cfd2-b86c-4701-9427-e6f881193695 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:31.170325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10cfd2-b86c-4701-9427-e6f881193695 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:31.170702 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10cfd2-b86c-4701-9427-e6f881193695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:31.208216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-6d10cfd2-b86c-4701-9427-e6f881193695 None None] MFA Rules not processed for user `fb77bfeec5c14d6db3f7a5eb71a6d79f`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:31.209802 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6d10cfd2-b86c-4701-9427-e6f881193695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:31.256690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:31.319977 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1187/2399] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:31.330460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc2877-83b0-4028-a546-a7b420a38966 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:31.332281 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b3fc2877-83b0-4028-a546-a7b420a38966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:31.430364 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc2877-83b0-4028-a546-a7b420a38966 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:31.436524 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d00a4-46b0-4d93-8fbd-276b160ae64d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:31.437155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d00a4-46b0-4d93-8fbd-276b160ae64d service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:31.437432 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d00a4-46b0-4d93-8fbd-276b160ae64d service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:31.437704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d00a4-46b0-4d93-8fbd-276b160ae64d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:31.451013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d00a4-46b0-4d93-8fbd-276b160ae64d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:31.455645 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d00a4-46b0-4d93-8fbd-276b160ae64d service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:31.487431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1213/2400] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:32.987894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac68ba-074e-4614-a696-c06c9f9af5b5 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:32.988904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac68ba-074e-4614-a696-c06c9f9af5b5 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:32.989979 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac68ba-074e-4614-a696-c06c9f9af5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:33.016257 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-d0ac68ba-074e-4614-a696-c06c9f9af5b5 None None] MFA Rules not processed for user `fb77bfeec5c14d6db3f7a5eb71a6d79f`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:33.017609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d0ac68ba-074e-4614-a696-c06c9f9af5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:33.030921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1188/2401] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:41.473055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:43.007322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=17 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:45.241069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08c8dd-5bfa-4325-a530-37f2733b52c6 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:45.241608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08c8dd-5bfa-4325-a530-37f2733b52c6 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:45.243357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08c8dd-5bfa-4325-a530-37f2733b52c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:45.264544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-1f08c8dd-5bfa-4325-a530-37f2733b52c6 None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:45.266120 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1f08c8dd-5bfa-4325-a530-37f2733b52c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:45.329690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1214/2402] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:56:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:45.342451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac68ba-074e-4614-a696-c06c9f9af5b5 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:45.346450 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d0ac68ba-074e-4614-a696-c06c9f9af5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:45.416563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac68ba-074e-4614-a696-c06c9f9af5b5 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:45.421521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a8dd1-b32a-42e6-a82f-7c6d0663602a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:45.422147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a8dd1-b32a-42e6-a82f-7c6d0663602a service swift] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:45.422453 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a8dd1-b32a-42e6-a82f-7c6d0663602a service swift] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:45.422755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a8dd1-b32a-42e6-a82f-7c6d0663602a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:45.434960 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a8dd1-b32a-42e6-a82f-7c6d0663602a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:45.437148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a8dd1-b32a-42e6-a82f-7c6d0663602a service swift] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:45.448587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1189/2403] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:56:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:56:51.101501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08c8dd-5bfa-4325-a530-37f2733b52c6 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:51.106652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1f08c8dd-5bfa-4325-a530-37f2733b52c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:51.170593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08c8dd-5bfa-4325-a530-37f2733b52c6 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:51.183239 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c7bf9-ca1b-413e-b2aa-d7f2540c0ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:51.184513 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c7bf9-ca1b-413e-b2aa-d7f2540c0ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:51.185097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c7bf9-ca1b-413e-b2aa-d7f2540c0ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:51.185671 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c7bf9-ca1b-413e-b2aa-d7f2540c0ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:51.188572 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-512c7bf9-ca1b-413e-b2aa-d7f2540c0ea8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:51.304927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c7bf9-ca1b-413e-b2aa-d7f2540c0ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:51.307055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c7bf9-ca1b-413e-b2aa-d7f2540c0ea8 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:51.319745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1215/2404] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:55.442475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=32 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:56.366827 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a8dd1-b32a-42e6-a82f-7c6d0663602a service swift] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:56.368344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-172a8dd1-b32a-42e6-a82f-7c6d0663602a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:56.438855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c7bf9-ca1b-413e-b2aa-d7f2540c0ea8 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:56.440406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-512c7bf9-ca1b-413e-b2aa-d7f2540c0ea8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:56.458805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c7bf9-ca1b-413e-b2aa-d7f2540c0ea8 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:56.465610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7c45a-8498-422b-9bd7-f94ec5fb6d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:56.467788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7c45a-8498-422b-9bd7-f94ec5fb6d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:56.467788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7c45a-8498-422b-9bd7-f94ec5fb6d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:56.467788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7c45a-8498-422b-9bd7-f94ec5fb6d13 admin admin] PATH_INFO: `/v3/users/074f3ddb689c4a60a05d3267080d7e7a` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:56.468588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece7c45a-8498-422b-9bd7-f94ec5fb6d13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=074f3ddb689c4a60a05d3267080d7e7a)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:56.470530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece7c45a-8498-422b-9bd7-f94ec5fb6d13 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:56.488394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a8dd1-b32a-42e6-a82f-7c6d0663602a service swift] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:56.498029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a786cb-8190-4bef-b293-b06e1591b93a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:56.498477 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a786cb-8190-4bef-b293-b06e1591b93a service placement] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:56.498950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a786cb-8190-4bef-b293-b06e1591b93a service placement] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:56.499231 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a786cb-8190-4bef-b293-b06e1591b93a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:56.512611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-57a786cb-8190-4bef-b293-b06e1591b93a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:56.529680 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a786cb-8190-4bef-b293-b06e1591b93a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:56.531776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a786cb-8190-4bef-b293-b06e1591b93a service placement] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:56.543619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1190/2405] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:56:56.549137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ece7c45a-8498-422b-9bd7-f94ec5fb6d13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '074f3ddb689c4a60a05d3267080d7e7a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:56.554106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:56.570457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ece7c45a-8498-422b-9bd7-f94ec5fb6d13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '074f3ddb689c4a60a05d3267080d7e7a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:56.578993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ece7c45a-8498-422b-9bd7-f94ec5fb6d13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '074f3ddb689c4a60a05d3267080d7e7a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:56.579485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ece7c45a-8498-422b-9bd7-f94ec5fb6d13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '074f3ddb689c4a60a05d3267080d7e7a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:56.580098 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ece7c45a-8498-422b-9bd7-f94ec5fb6d13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '074f3ddb689c4a60a05d3267080d7e7a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:56.594220 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1216/2406] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:56:56 2022] DELETE /identity/v3/users/074f3ddb689c4a60a05d3267080d7e7a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:56.803573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a786cb-8190-4bef-b293-b06e1591b93a service placement] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:56.819959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a786cb-8190-4bef-b293-b06e1591b93a service placement] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:56.880332 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:56.880967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:56.881243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:56.881510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3 admin admin] PATH_INFO: `/v3/projects/21db26f543ab4a67b932184ba810f4e9` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:56.882750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21db26f543ab4a67b932184ba810f4e9)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:56.884852 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:56.934464 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21db26f543ab4a67b932184ba810f4e9', 'request_id': 'req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:56.940846 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21db26f543ab4a67b932184ba810f4e9', 'request_id': 'req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:56.948523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21db26f543ab4a67b932184ba810f4e9', 'request_id': 'req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:56.960294 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3 admin admin] The token cache is being invalidate because project 21db26f543ab4a67b932184ba810f4e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:56:56.960524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:56.961142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-96f0ec73-c71c-4d43-aa72-e2ef0a417eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:56:56.962771 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1191/2407] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:56:56 2022] DELETE /identity/v3/projects/21db26f543ab4a67b932184ba810f4e9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:56.967869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8388341-1170-498d-83b7-b161a8e5f46b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:56.968154 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8388341-1170-498d-83b7-b161a8e5f46b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:56.968434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8388341-1170-498d-83b7-b161a8e5f46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:56.989223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e8388341-1170-498d-83b7-b161a8e5f46b None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:56.990525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e8388341-1170-498d-83b7-b161a8e5f46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:57.056603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1217/2408] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:57.064547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dafc19-e5b4-4bbe-9088-1e752897d841 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:57.064689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dafc19-e5b4-4bbe-9088-1e752897d841 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:57.064993 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dafc19-e5b4-4bbe-9088-1e752897d841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:57.086336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-78dafc19-e5b4-4bbe-9088-1e752897d841 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:57.087583 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-78dafc19-e5b4-4bbe-9088-1e752897d841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:57.095518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1192/2409] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:57.100506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8388341-1170-498d-83b7-b161a8e5f46b None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:57.110517 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8388341-1170-498d-83b7-b161a8e5f46b None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:57.115962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb37ae-b6fd-4e47-acab-43ee02b36358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:57.116576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb37ae-b6fd-4e47-acab-43ee02b36358 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:57.116852 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb37ae-b6fd-4e47-acab-43ee02b36358 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:57.117150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb37ae-b6fd-4e47-acab-43ee02b36358 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:57.117854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb37ae-b6fd-4e47-acab-43ee02b36358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:56:57.118160 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb37ae-b6fd-4e47-acab-43ee02b36358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:57.119912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb37ae-b6fd-4e47-acab-43ee02b36358 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:57.132532 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1218/2410] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:56:57.136904 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dafc19-e5b4-4bbe-9088-1e752897d841 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:57.146378 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dafc19-e5b4-4bbe-9088-1e752897d841 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:57.151545 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612319cc-c505-4723-9b0b-7a2a9faad86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:57.152165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612319cc-c505-4723-9b0b-7a2a9faad86f admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:57.152448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612319cc-c505-4723-9b0b-7a2a9faad86f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:57.152724 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612319cc-c505-4723-9b0b-7a2a9faad86f admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:57.153432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612319cc-c505-4723-9b0b-7a2a9faad86f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:57.155377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612319cc-c505-4723-9b0b-7a2a9faad86f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:57.168010 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:57.186966 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1193/2411] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:56:57 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:56:57.191242 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb37ae-b6fd-4e47-acab-43ee02b36358 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:57.202333 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb37ae-b6fd-4e47-acab-43ee02b36358 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:57.263242 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dc442-0fcb-48e7-abe9-4edc0617e938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:57.263874 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc442-0fcb-48e7-abe9-4edc0617e938 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:57.264155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc442-0fcb-48e7-abe9-4edc0617e938 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:57.264428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc442-0fcb-48e7-abe9-4edc0617e938 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:57.265090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742dc442-0fcb-48e7-abe9-4edc0617e938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:57.267065 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742dc442-0fcb-48e7-abe9-4edc0617e938 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:57.291225 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1219/2412] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:56:57 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:56:57.294653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612319cc-c505-4723-9b0b-7a2a9faad86f admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:57.306504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612319cc-c505-4723-9b0b-7a2a9faad86f admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:57.311500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a14d0a-e1da-4781-922c-212f134d6184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:57.312122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a14d0a-e1da-4781-922c-212f134d6184 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:57.312402 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a14d0a-e1da-4781-922c-212f134d6184 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:57.312675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a14d0a-e1da-4781-922c-212f134d6184 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:57.313370 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a14d0a-e1da-4781-922c-212f134d6184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:57.315017 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a14d0a-e1da-4781-922c-212f134d6184 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:57.329159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1194/2413] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:56:57.333094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dc442-0fcb-48e7-abe9-4edc0617e938 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:57.343886 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dc442-0fcb-48e7-abe9-4edc0617e938 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:57.353472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b77138-86c1-4127-aec6-c428bb1f10cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:57.354117 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b77138-86c1-4127-aec6-c428bb1f10cb admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:57.354399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b77138-86c1-4127-aec6-c428bb1f10cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:57.354721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b77138-86c1-4127-aec6-c428bb1f10cb admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:57.355416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b77138-86c1-4127-aec6-c428bb1f10cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:57.357690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b77138-86c1-4127-aec6-c428bb1f10cb admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:57.371773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1220/2414] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:56:57.376346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a14d0a-e1da-4781-922c-212f134d6184 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:57.392512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a14d0a-e1da-4781-922c-212f134d6184 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:57.398276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d32ec-9104-4861-bff5-72d9515d30b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:57.399203 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d32ec-9104-4861-bff5-72d9515d30b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:57.399592 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d32ec-9104-4861-bff5-72d9515d30b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:57.400150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d32ec-9104-4861-bff5-72d9515d30b4 admin admin] PATH_INFO: `/v3/projects/c37bc978cd454fee913fefa5e186cedf/users/17319125d83f44169ffeb01e8e843f2e/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:57.422052 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d32ec-9104-4861-bff5-72d9515d30b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c37bc978cd454fee913fefa5e186cedf, user_id=17319125d83f44169ffeb01e8e843f2e, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:57.424887 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d32ec-9104-4861-bff5-72d9515d30b4 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:57.453948 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1195/2415] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:56:57 2022] PUT /identity/v3/projects/c37bc978cd454fee913fefa5e186cedf/users/17319125d83f44169ffeb01e8e843f2e/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:56:57.456166 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bb70a-e5be-43b2-b806-7bcf532bbe47 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:57.456636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bb70a-e5be-43b2-b806-7bcf532bbe47 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:57.456898 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bb70a-e5be-43b2-b806-7bcf532bbe47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:57.496403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-3b4bb70a-e5be-43b2-b806-7bcf532bbe47 None None] MFA Rules not processed for user `17319125d83f44169ffeb01e8e843f2e`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:56:57.497816 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3b4bb70a-e5be-43b2-b806-7bcf532bbe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:57.521856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:56:57.572763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1221/2416] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:56:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:56:57.584285 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d32ec-9104-4861-bff5-72d9515d30b4 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:56:57.586074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3a8d32ec-9104-4861-bff5-72d9515d30b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:56:57.658822 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d32ec-9104-4861-bff5-72d9515d30b4 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:56:57.663700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54d645-636c-4e8e-b4ba-068985cbd908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:56:57.664335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54d645-636c-4e8e-b4ba-068985cbd908 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:56:57.664614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54d645-636c-4e8e-b4ba-068985cbd908 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:56:57.664958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54d645-636c-4e8e-b4ba-068985cbd908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:56:57.677151 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54d645-636c-4e8e-b4ba-068985cbd908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:56:57.678361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54d645-636c-4e8e-b4ba-068985cbd908 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:56:57.690167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1196/2417] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:56:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:02.320546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b8903-0c3c-41ce-94c3-0e37576c106f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:02.321118 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b8903-0c3c-41ce-94c3-0e37576c106f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:02.321227 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b8903-0c3c-41ce-94c3-0e37576c106f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:02.342352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-4f6b8903-0c3c-41ce-94c3-0e37576c106f None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:02.343649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4f6b8903-0c3c-41ce-94c3-0e37576c106f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:02.429519 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1222/2418] 10.222.0.86 () {58 vars in 967 bytes} [Fri Aug 5 19:57:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:02.444579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54d645-636c-4e8e-b4ba-068985cbd908 service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:02.446610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3a54d645-636c-4e8e-b4ba-068985cbd908 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:02.565237 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54d645-636c-4e8e-b4ba-068985cbd908 service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:02.579508 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a56bc3-fe5a-468b-8413-c7f83eec1f02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:02.580375 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a56bc3-fe5a-468b-8413-c7f83eec1f02 service swift] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:02.580651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a56bc3-fe5a-468b-8413-c7f83eec1f02 service swift] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:02.580920 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a56bc3-fe5a-468b-8413-c7f83eec1f02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:02.590513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a56bc3-fe5a-468b-8413-c7f83eec1f02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:02.592167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a56bc3-fe5a-468b-8413-c7f83eec1f02 service swift] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:02.602295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1197/2419] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:57:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:57:03.800345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce4140-2de5-4fdf-849f-169c86ee9c5f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:03.800869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce4140-2de5-4fdf-849f-169c86ee9c5f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:03.800869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce4140-2de5-4fdf-849f-169c86ee9c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:03.823092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-70ce4140-2de5-4fdf-849f-169c86ee9c5f None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:03.824323 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-70ce4140-2de5-4fdf-849f-169c86ee9c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:03.831958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1223/2420] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:57:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:03.844315 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a56bc3-fe5a-468b-8413-c7f83eec1f02 service swift] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:03.855334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a56bc3-fe5a-468b-8413-c7f83eec1f02 service swift] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:03.860750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7310ee-8c56-4a8d-a1ba-474c50025591 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:03.861374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7310ee-8c56-4a8d-a1ba-474c50025591 service swift] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:03.861649 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7310ee-8c56-4a8d-a1ba-474c50025591 service swift] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:03.861915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7310ee-8c56-4a8d-a1ba-474c50025591 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:03.880201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7310ee-8c56-4a8d-a1ba-474c50025591 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:03.881822 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7310ee-8c56-4a8d-a1ba-474c50025591 service swift] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:03.894309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1198/2421] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:57:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:57:04.467327 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbed93-a651-4530-ae07-295a331f4ecb None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:04.467610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbed93-a651-4530-ae07-295a331f4ecb None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:04.467887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbed93-a651-4530-ae07-295a331f4ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:04.492028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-15dbed93-a651-4530-ae07-295a331f4ecb None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:04.493264 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-15dbed93-a651-4530-ae07-295a331f4ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:04.500753 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1224/2422] 10.222.0.86 () {58 vars in 967 bytes} [Fri Aug 5 19:57:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:04.514393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7310ee-8c56-4a8d-a1ba-474c50025591 service swift] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:04.526706 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7310ee-8c56-4a8d-a1ba-474c50025591 service swift] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:04.532022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc760e-ee5a-4c1b-ae8d-79341a79597f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:04.532639 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc760e-ee5a-4c1b-ae8d-79341a79597f service swift] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:04.532916 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc760e-ee5a-4c1b-ae8d-79341a79597f service swift] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:04.533191 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc760e-ee5a-4c1b-ae8d-79341a79597f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:04.543063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc760e-ee5a-4c1b-ae8d-79341a79597f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:04.544815 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc760e-ee5a-4c1b-ae8d-79341a79597f service swift] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:04.555053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1199/2423] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:57:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:57:04.788267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b9642-71d7-4267-a9be-0793b19bda6e None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:04.788550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b9642-71d7-4267-a9be-0793b19bda6e None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:04.788846 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b9642-71d7-4267-a9be-0793b19bda6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:04.810799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-6c8b9642-71d7-4267-a9be-0793b19bda6e None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:04.812025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6c8b9642-71d7-4267-a9be-0793b19bda6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:04.820116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1225/2424] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:57:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:04.832355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc760e-ee5a-4c1b-ae8d-79341a79597f service swift] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:04.844044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc760e-ee5a-4c1b-ae8d-79341a79597f service swift] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:04.850476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2dad1-c8bf-4efc-b619-3f452927d3d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:04.851218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2dad1-c8bf-4efc-b619-3f452927d3d1 service swift] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:04.851218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2dad1-c8bf-4efc-b619-3f452927d3d1 service swift] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:04.851498 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2dad1-c8bf-4efc-b619-3f452927d3d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:04.860565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2dad1-c8bf-4efc-b619-3f452927d3d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:04.862162 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2dad1-c8bf-4efc-b619-3f452927d3d1 service swift] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:04.871984 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1200/2425] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:57:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:57:05.223037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb9e64-3880-45e1-94a2-94efe084c1c9 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:05.223264 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb9e64-3880-45e1-94a2-94efe084c1c9 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:05.223627 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb9e64-3880-45e1-94a2-94efe084c1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:05.244433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-06fb9e64-3880-45e1-94a2-94efe084c1c9 None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:05.245679 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-06fb9e64-3880-45e1-94a2-94efe084c1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:05.252993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1226/2426] 10.222.0.86 () {58 vars in 967 bytes} [Fri Aug 5 19:57:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:05.265903 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2dad1-c8bf-4efc-b619-3f452927d3d1 service swift] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:05.277627 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2dad1-c8bf-4efc-b619-3f452927d3d1 service swift] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:05.282855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70748827-d6d4-46f2-9792-7d4b7f6da49a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:05.283481 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70748827-d6d4-46f2-9792-7d4b7f6da49a service swift] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:05.283745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70748827-d6d4-46f2-9792-7d4b7f6da49a service swift] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:05.284024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70748827-d6d4-46f2-9792-7d4b7f6da49a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:05.290648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:05.294159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70748827-d6d4-46f2-9792-7d4b7f6da49a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:05.296170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70748827-d6d4-46f2-9792-7d4b7f6da49a service swift] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:05.309006 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1201/2427] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:57:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:57:06.562877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb9e64-3880-45e1-94a2-94efe084c1c9 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:06.641449 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb9e64-3880-45e1-94a2-94efe084c1c9 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:06.647184 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:06.647884 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:06.648168 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:06.648483 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] PATH_INFO: `/v3/users/17319125d83f44169ffeb01e8e843f2e` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:06.649791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=17319125d83f44169ffeb01e8e843f2e)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:06.651836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:06.720144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:06.742394 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17319125d83f44169ffeb01e8e843f2e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:06.765147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17319125d83f44169ffeb01e8e843f2e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:06.777576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17319125d83f44169ffeb01e8e843f2e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:06.778672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17319125d83f44169ffeb01e8e843f2e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:06.779576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17319125d83f44169ffeb01e8e843f2e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:06.797853 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1227/2428] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:57:06 2022] DELETE /identity/v3/users/17319125d83f44169ffeb01e8e843f2e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:06.803573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9be8c-01c5-4734-a462-3ee2caabc8ac None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:06.804145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9be8c-01c5-4734-a462-3ee2caabc8ac None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:06.804145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9be8c-01c5-4734-a462-3ee2caabc8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:06.823510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-dce9be8c-01c5-4734-a462-3ee2caabc8ac None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:06.824744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-dce9be8c-01c5-4734-a462-3ee2caabc8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:06.889669 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1202/2429] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:06.900210 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:06.901919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:06.961905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2416747-bc9c-4e10-8b68-df3a53425d8d admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:06.965657 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b8da1-2b5e-4f06-825a-8b0661b3f10a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:06.966280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b8da1-2b5e-4f06-825a-8b0661b3f10a service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:06.966556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b8da1-2b5e-4f06-825a-8b0661b3f10a service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:06.966883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b8da1-2b5e-4f06-825a-8b0661b3f10a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:06.977363 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b8da1-2b5e-4f06-825a-8b0661b3f10a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:06.978982 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b8da1-2b5e-4f06-825a-8b0661b3f10a service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:06.989155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1228/2430] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:07.295849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9be8c-01c5-4734-a462-3ee2caabc8ac None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:07.297357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-dce9be8c-01c5-4734-a462-3ee2caabc8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:07.312516 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9be8c-01c5-4734-a462-3ee2caabc8ac None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:07.320304 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:07.321156 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:07.321550 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:07.321952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6 admin admin] PATH_INFO: `/v3/projects/c37bc978cd454fee913fefa5e186cedf` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:07.323469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c37bc978cd454fee913fefa5e186cedf)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:07.325587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:07.400376 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c37bc978cd454fee913fefa5e186cedf', 'request_id': 'req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:07.408826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c37bc978cd454fee913fefa5e186cedf', 'request_id': 'req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:07.417757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c37bc978cd454fee913fefa5e186cedf', 'request_id': 'req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:07.430329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6 admin admin] The token cache is being invalidate because project c37bc978cd454fee913fefa5e186cedf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:57:07.430748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:07.431419 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-17c842e9-21c5-4b6b-b9ac-bbede38b1fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:07.432981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1203/2431] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:57:07 2022] DELETE /identity/v3/projects/c37bc978cd454fee913fefa5e186cedf => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:07.437700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cbc21-c54b-4c21-9270-ea03b22d6eb9 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:07.438024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cbc21-c54b-4c21-9270-ea03b22d6eb9 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:07.438271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cbc21-c54b-4c21-9270-ea03b22d6eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:07.465677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-b60cbc21-c54b-4c21-9270-ea03b22d6eb9 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:07.467229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b60cbc21-c54b-4c21-9270-ea03b22d6eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:07.539140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1229/2432] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:07.546860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985dbde-eced-4874-b5c4-cc9ae062b843 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:07.547190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985dbde-eced-4874-b5c4-cc9ae062b843 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:07.547463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985dbde-eced-4874-b5c4-cc9ae062b843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:07.574949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-3985dbde-eced-4874-b5c4-cc9ae062b843 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:07.576272 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3985dbde-eced-4874-b5c4-cc9ae062b843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:07.577403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cbc21-c54b-4c21-9270-ea03b22d6eb9 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:07.578840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b60cbc21-c54b-4c21-9270-ea03b22d6eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:07.584737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1204/2433] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:07.588736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985dbde-eced-4874-b5c4-cc9ae062b843 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:07.591466 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cbc21-c54b-4c21-9270-ea03b22d6eb9 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:07.595974 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf391ce-fb0a-4b8a-bb4a-913ce9947b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:07.596599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf391ce-fb0a-4b8a-bb4a-913ce9947b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:07.596901 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf391ce-fb0a-4b8a-bb4a-913ce9947b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:07.597138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf391ce-fb0a-4b8a-bb4a-913ce9947b80 admin admin] PATH_INFO: `/v3/users/fb77bfeec5c14d6db3f7a5eb71a6d79f` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:07.597867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985dbde-eced-4874-b5c4-cc9ae062b843 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:07.598329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf391ce-fb0a-4b8a-bb4a-913ce9947b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb77bfeec5c14d6db3f7a5eb71a6d79f)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:07.600947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf391ce-fb0a-4b8a-bb4a-913ce9947b80 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:07.602720 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeefad3-c4e7-44f6-8b5b-239104034b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:07.603314 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeefad3-c4e7-44f6-8b5b-239104034b3a admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:07.603590 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeefad3-c4e7-44f6-8b5b-239104034b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:07.603860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeefad3-c4e7-44f6-8b5b-239104034b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:07.604549 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeefad3-c4e7-44f6-8b5b-239104034b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:57:07.604848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeefad3-c4e7-44f6-8b5b-239104034b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:07.606414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeefad3-c4e7-44f6-8b5b-239104034b3a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:07.618893 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1205/2434] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:57:07.623303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeefad3-c4e7-44f6-8b5b-239104034b3a admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:07.635043 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeefad3-c4e7-44f6-8b5b-239104034b3a admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:07.640276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669139e-bb06-49f3-8475-3d7e5676b057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:07.640909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669139e-bb06-49f3-8475-3d7e5676b057 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:07.641185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669139e-bb06-49f3-8475-3d7e5676b057 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:07.641451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669139e-bb06-49f3-8475-3d7e5676b057 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:07.642111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669139e-bb06-49f3-8475-3d7e5676b057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:07.644021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669139e-bb06-49f3-8475-3d7e5676b057 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:07.651214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-baf391ce-fb0a-4b8a-bb4a-913ce9947b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb77bfeec5c14d6db3f7a5eb71a6d79f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:07.659034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:07.663654 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-baf391ce-fb0a-4b8a-bb4a-913ce9947b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb77bfeec5c14d6db3f7a5eb71a6d79f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:07.672068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-baf391ce-fb0a-4b8a-bb4a-913ce9947b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb77bfeec5c14d6db3f7a5eb71a6d79f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:07.673412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1206/2435] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:07 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:07.673556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-baf391ce-fb0a-4b8a-bb4a-913ce9947b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb77bfeec5c14d6db3f7a5eb71a6d79f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:07.674781 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-baf391ce-fb0a-4b8a-bb4a-913ce9947b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb77bfeec5c14d6db3f7a5eb71a6d79f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:07.678316 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669139e-bb06-49f3-8475-3d7e5676b057 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:07.679933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2669139e-bb06-49f3-8475-3d7e5676b057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:07.690123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1230/2436] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:57:07 2022] DELETE /identity/v3/users/fb77bfeec5c14d6db3f7a5eb71a6d79f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:57:07.695176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c878a22-4048-4065-8df9-7f59e19debe1 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:07.695465 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c878a22-4048-4065-8df9-7f59e19debe1 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:07.695753 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c878a22-4048-4065-8df9-7f59e19debe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:07.714724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-7c878a22-4048-4065-8df9-7f59e19debe1 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:07.715929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7c878a22-4048-4065-8df9-7f59e19debe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:07.722533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:07.748156 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669139e-bb06-49f3-8475-3d7e5676b057 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:07.778071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1231/2437] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:57:07.787977 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c878a22-4048-4065-8df9-7f59e19debe1 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:07.789392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7c878a22-4048-4065-8df9-7f59e19debe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:07.815116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e658205-c602-401b-b3a7-bd16d26d7230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:07.815911 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e658205-c602-401b-b3a7-bd16d26d7230 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:07.816327 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e658205-c602-401b-b3a7-bd16d26d7230 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:07.816721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e658205-c602-401b-b3a7-bd16d26d7230 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:07.818255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e658205-c602-401b-b3a7-bd16d26d7230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:07.820895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e658205-c602-401b-b3a7-bd16d26d7230 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:07.848017 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1207/2438] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:07 2022] POST /identity/v3/users => generated 291 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:57:07.858109 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c878a22-4048-4065-8df9-7f59e19debe1 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:07.861944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e658205-c602-401b-b3a7-bd16d26d7230 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:07.866728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50960e75-71b4-439c-a796-07cf0906a52e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:07.867369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50960e75-71b4-439c-a796-07cf0906a52e service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:07.867647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50960e75-71b4-439c-a796-07cf0906a52e service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:07.867919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50960e75-71b4-439c-a796-07cf0906a52e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:07.875033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e658205-c602-401b-b3a7-bd16d26d7230 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:07.878317 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50960e75-71b4-439c-a796-07cf0906a52e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:07.880453 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50960e75-71b4-439c-a796-07cf0906a52e service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:07.882294 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994618a-d2af-459d-81aa-d3b6fc09f7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:07.882963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994618a-d2af-459d-81aa-d3b6fc09f7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:07.883248 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994618a-d2af-459d-81aa-d3b6fc09f7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:07.884910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994618a-d2af-459d-81aa-d3b6fc09f7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:07.884910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994618a-d2af-459d-81aa-d3b6fc09f7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:07.885908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994618a-d2af-459d-81aa-d3b6fc09f7e6 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:07.891324 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1232/2439] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:57:07.897802 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1208/2440] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:57:07.901842 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50960e75-71b4-439c-a796-07cf0906a52e service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:07.912636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50960e75-71b4-439c-a796-07cf0906a52e service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:07.919029 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70035733-d2a3-4902-b59a-6031d88e0759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:07.919854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035733-d2a3-4902-b59a-6031d88e0759 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:07.920331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035733-d2a3-4902-b59a-6031d88e0759 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:07.920811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70035733-d2a3-4902-b59a-6031d88e0759 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:07.922056 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70035733-d2a3-4902-b59a-6031d88e0759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:07.925018 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70035733-d2a3-4902-b59a-6031d88e0759 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:07.936627 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1233/2441] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:57:07.940591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994618a-d2af-459d-81aa-d3b6fc09f7e6 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:07.951089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994618a-d2af-459d-81aa-d3b6fc09f7e6 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:07.966307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1d0a6-bfab-498f-9d30-da0e0fc09148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:07.967026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1d0a6-bfab-498f-9d30-da0e0fc09148 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:07.967325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1d0a6-bfab-498f-9d30-da0e0fc09148 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:07.967567 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1d0a6-bfab-498f-9d30-da0e0fc09148 admin admin] PATH_INFO: `/v3/projects/572551ada6db42bb868f576d27a5af75/users/174b2a2ff4474c2a81209c1a9418f77d/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:07.985852 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1d0a6-bfab-498f-9d30-da0e0fc09148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=572551ada6db42bb868f576d27a5af75, user_id=174b2a2ff4474c2a81209c1a9418f77d, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:07.988614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1d0a6-bfab-498f-9d30-da0e0fc09148 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.005943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1209/2442] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:07 2022] PUT /identity/v3/projects/572551ada6db42bb868f576d27a5af75/users/174b2a2ff4474c2a81209c1a9418f77d/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:08.016894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d7d1c-1bd7-43cb-91fb-c837949c981c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.017233 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d7d1c-1bd7-43cb-91fb-c837949c981c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.017594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d7d1c-1bd7-43cb-91fb-c837949c981c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.054649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-693d7d1c-1bd7-43cb-91fb-c837949c981c None None] MFA Rules not processed for user `174b2a2ff4474c2a81209c1a9418f77d`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:08.055959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-693d7d1c-1bd7-43cb-91fb-c837949c981c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:08.104356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1d0a6-bfab-498f-9d30-da0e0fc09148 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.106205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-11b1d0a6-bfab-498f-9d30-da0e0fc09148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:08.138363 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:08.148574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1234/2443] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:08.151664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d7d1c-1bd7-43cb-91fb-c837949c981c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.200214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1d0a6-bfab-498f-9d30-da0e0fc09148 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.208359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe1b0f-7262-4112-97f8-af1c9eba7795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.209073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe1b0f-7262-4112-97f8-af1c9eba7795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.209363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe1b0f-7262-4112-97f8-af1c9eba7795 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.210670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe1b0f-7262-4112-97f8-af1c9eba7795 admin admin] PATH_INFO: `/v3/projects/9a5122857f764dcf977389f772000b4e` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.210952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe1b0f-7262-4112-97f8-af1c9eba7795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a5122857f764dcf977389f772000b4e)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.213432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe1b0f-7262-4112-97f8-af1c9eba7795 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.250524 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d7d1c-1bd7-43cb-91fb-c837949c981c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.256011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be44bb-b405-438c-9dde-ea49adfe275b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.256713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be44bb-b405-438c-9dde-ea49adfe275b admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.257054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be44bb-b405-438c-9dde-ea49adfe275b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.257409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be44bb-b405-438c-9dde-ea49adfe275b admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.258136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be44bb-b405-438c-9dde-ea49adfe275b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.260253 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be44bb-b405-438c-9dde-ea49adfe275b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.274059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:08.282123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-cdfe1b0f-7262-4112-97f8-af1c9eba7795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a5122857f764dcf977389f772000b4e', 'request_id': 'req-cdfe1b0f-7262-4112-97f8-af1c9eba7795'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:08.292255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-cdfe1b0f-7262-4112-97f8-af1c9eba7795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a5122857f764dcf977389f772000b4e', 'request_id': 'req-cdfe1b0f-7262-4112-97f8-af1c9eba7795'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:08.300850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-cdfe1b0f-7262-4112-97f8-af1c9eba7795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a5122857f764dcf977389f772000b4e', 'request_id': 'req-cdfe1b0f-7262-4112-97f8-af1c9eba7795'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:08.302521 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1235/2444] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:08 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:57:08.306495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be44bb-b405-438c-9dde-ea49adfe275b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.320305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-cdfe1b0f-7262-4112-97f8-af1c9eba7795 admin admin] The token cache is being invalidate because project 9a5122857f764dcf977389f772000b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:57:08.320602 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-cdfe1b0f-7262-4112-97f8-af1c9eba7795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:08.321475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-cdfe1b0f-7262-4112-97f8-af1c9eba7795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:08.323796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1210/2445] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:57:08 2022] DELETE /identity/v3/projects/9a5122857f764dcf977389f772000b4e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:57:08.328509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09843ac-ca22-45a1-9d77-a5227a793383 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.328816 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09843ac-ca22-45a1-9d77-a5227a793383 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.329098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09843ac-ca22-45a1-9d77-a5227a793383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.354970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c09843ac-ca22-45a1-9d77-a5227a793383 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:08.356217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c09843ac-ca22-45a1-9d77-a5227a793383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:08.384665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be44bb-b405-438c-9dde-ea49adfe275b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.389742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d3d46-cebe-4c7e-b063-b62b0b9a36b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.390360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d3d46-cebe-4c7e-b063-b62b0b9a36b3 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.390648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d3d46-cebe-4c7e-b063-b62b0b9a36b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.390936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d3d46-cebe-4c7e-b063-b62b0b9a36b3 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.391631 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d3d46-cebe-4c7e-b063-b62b0b9a36b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.393644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d3d46-cebe-4c7e-b063-b62b0b9a36b3 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.422884 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1236/2446] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:08 2022] POST /identity/v3/users => generated 296 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:08.423994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d3d46-cebe-4c7e-b063-b62b0b9a36b3 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.425604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-266d3d46-cebe-4c7e-b063-b62b0b9a36b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:08.427445 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1211/2447] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:57:08.435480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d5ee0-c9cd-49b3-950d-2f4eb135fd3a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.435770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d5ee0-c9cd-49b3-950d-2f4eb135fd3a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.436068 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d5ee0-c9cd-49b3-950d-2f4eb135fd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.441356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d3d46-cebe-4c7e-b063-b62b0b9a36b3 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.448157 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48152b-bb86-4b27-a36b-7dc022240e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.448831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48152b-bb86-4b27-a36b-7dc022240e01 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.449403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48152b-bb86-4b27-a36b-7dc022240e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.449403 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48152b-bb86-4b27-a36b-7dc022240e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.450073 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48152b-bb86-4b27-a36b-7dc022240e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.451755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48152b-bb86-4b27-a36b-7dc022240e01 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.461695 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-ab5d5ee0-c9cd-49b3-950d-2f4eb135fd3a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:08.463681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ab5d5ee0-c9cd-49b3-950d-2f4eb135fd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:08.464734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1237/2448] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:57:08.468841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48152b-bb86-4b27-a36b-7dc022240e01 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.479749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1212/2449] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:57:08.479749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d5ee0-c9cd-49b3-950d-2f4eb135fd3a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.486499 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48152b-bb86-4b27-a36b-7dc022240e01 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.490772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d5ee0-c9cd-49b3-950d-2f4eb135fd3a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.492990 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85a543-b3e6-4bce-970d-af63217139a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.493704 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85a543-b3e6-4bce-970d-af63217139a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.493984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85a543-b3e6-4bce-970d-af63217139a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.494274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85a543-b3e6-4bce-970d-af63217139a4 admin admin] PATH_INFO: `/v3/projects/0ca9b7283d6543faa6a9b1f01fea420b/users/6af12b3fa05447b88457651a108e93fb/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.495891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77420bbc-2118-417f-9ac0-339ea9f49795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.496572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77420bbc-2118-417f-9ac0-339ea9f49795 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.496934 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77420bbc-2118-417f-9ac0-339ea9f49795 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.497261 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77420bbc-2118-417f-9ac0-339ea9f49795 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.498022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77420bbc-2118-417f-9ac0-339ea9f49795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:57:08.498478 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77420bbc-2118-417f-9ac0-339ea9f49795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.500399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77420bbc-2118-417f-9ac0-339ea9f49795 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.514466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1213/2450] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:57:08.517165 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85a543-b3e6-4bce-970d-af63217139a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca9b7283d6543faa6a9b1f01fea420b, user_id=6af12b3fa05447b88457651a108e93fb, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.518865 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77420bbc-2118-417f-9ac0-339ea9f49795 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.520805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85a543-b3e6-4bce-970d-af63217139a4 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.530290 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77420bbc-2118-417f-9ac0-339ea9f49795 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.537481 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30c1df-b23b-41d7-874a-5dcf03c42bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.538544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30c1df-b23b-41d7-874a-5dcf03c42bcd admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.539018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30c1df-b23b-41d7-874a-5dcf03c42bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.539293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30c1df-b23b-41d7-874a-5dcf03c42bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.539744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1238/2451] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:08 2022] PUT /identity/v3/projects/0ca9b7283d6543faa6a9b1f01fea420b/users/6af12b3fa05447b88457651a108e93fb/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:57:08.540008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30c1df-b23b-41d7-874a-5dcf03c42bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.542424 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30c1df-b23b-41d7-874a-5dcf03c42bcd admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.543331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85a543-b3e6-4bce-970d-af63217139a4 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.584030 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1214/2452] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:08 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:57:08.588243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30c1df-b23b-41d7-874a-5dcf03c42bcd admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.595221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:08.627752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85a543-b3e6-4bce-970d-af63217139a4 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.676138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30c1df-b23b-41d7-874a-5dcf03c42bcd admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.682127 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70215efb-1104-4e12-b215-7b58ce016838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.682895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215efb-1104-4e12-b215-7b58ce016838 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.683185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215efb-1104-4e12-b215-7b58ce016838 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.683437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215efb-1104-4e12-b215-7b58ce016838 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.684090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70215efb-1104-4e12-b215-7b58ce016838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.686009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70215efb-1104-4e12-b215-7b58ce016838 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.704641 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f263ba-2110-40a5-a0ea-e05c3e7682c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.705257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f263ba-2110-40a5-a0ea-e05c3e7682c5 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.705531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f263ba-2110-40a5-a0ea-e05c3e7682c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.705823 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f263ba-2110-40a5-a0ea-e05c3e7682c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.706508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f263ba-2110-40a5-a0ea-e05c3e7682c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.709070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f263ba-2110-40a5-a0ea-e05c3e7682c5 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.709586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1215/2453] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:08 2022] POST /identity/v3/users => generated 290 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:57:08.713368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70215efb-1104-4e12-b215-7b58ce016838 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.722348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1239/2454] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:57:08.723213 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70215efb-1104-4e12-b215-7b58ce016838 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.724750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f263ba-2110-40a5-a0ea-e05c3e7682c5 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.731050 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26519cf6-af05-43be-a467-3b476e0c5266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.731798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26519cf6-af05-43be-a467-3b476e0c5266 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.732086 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26519cf6-af05-43be-a467-3b476e0c5266 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.732362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26519cf6-af05-43be-a467-3b476e0c5266 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.733073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26519cf6-af05-43be-a467-3b476e0c5266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.734715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26519cf6-af05-43be-a467-3b476e0c5266 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.735787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f263ba-2110-40a5-a0ea-e05c3e7682c5 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.741364 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5b46f-d85d-432a-ad1d-0209cfdc30f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.742023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5b46f-d85d-432a-ad1d-0209cfdc30f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.742301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5b46f-d85d-432a-ad1d-0209cfdc30f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.742569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5b46f-d85d-432a-ad1d-0209cfdc30f6 admin admin] PATH_INFO: `/v3/projects/0ca9b7283d6543faa6a9b1f01fea420b/users/6af12b3fa05447b88457651a108e93fb/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.744627 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb5b46f-d85d-432a-ad1d-0209cfdc30f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca9b7283d6543faa6a9b1f01fea420b, user_id=6af12b3fa05447b88457651a108e93fb, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.749395 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1216/2455] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:57:08.749538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb5b46f-d85d-432a-ad1d-0209cfdc30f6 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.750715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26519cf6-af05-43be-a467-3b476e0c5266 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.761823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26519cf6-af05-43be-a467-3b476e0c5266 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.765910 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:08.769105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1240/2456] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:08 2022] PUT /identity/v3/projects/0ca9b7283d6543faa6a9b1f01fea420b/users/6af12b3fa05447b88457651a108e93fb/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:57:08.781097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e902236-699a-44f2-922e-4890e758ae71 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.781374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e902236-699a-44f2-922e-4890e758ae71 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.781648 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e902236-699a-44f2-922e-4890e758ae71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.810344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-1e902236-699a-44f2-922e-4890e758ae71 None None] MFA Rules not processed for user `6af12b3fa05447b88457651a108e93fb`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:08.811595 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1e902236-699a-44f2-922e-4890e758ae71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:08.824705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77bfcb-0e3d-4f94-9302-6857f91ede41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.825883 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77bfcb-0e3d-4f94-9302-6857f91ede41 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.826169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77bfcb-0e3d-4f94-9302-6857f91ede41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.826442 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77bfcb-0e3d-4f94-9302-6857f91ede41 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.827169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77bfcb-0e3d-4f94-9302-6857f91ede41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.828763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77bfcb-0e3d-4f94-9302-6857f91ede41 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.839157 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1217/2457] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:08.843240 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77bfcb-0e3d-4f94-9302-6857f91ede41 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.853743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77bfcb-0e3d-4f94-9302-6857f91ede41 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.858494 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a815884-8dfd-4e40-b1c7-952c42d999c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:08.859182 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a815884-8dfd-4e40-b1c7-952c42d999c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.859464 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a815884-8dfd-4e40-b1c7-952c42d999c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.859738 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a815884-8dfd-4e40-b1c7-952c42d999c9 admin admin] PATH_INFO: `/v3/projects/7372aebdf6554202a45b0bbd7bad0700/users/19c2f15eb1644278b911cc50370da9b7/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.875509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a815884-8dfd-4e40-b1c7-952c42d999c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7372aebdf6554202a45b0bbd7bad0700, user_id=19c2f15eb1644278b911cc50370da9b7, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:08.878177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a815884-8dfd-4e40-b1c7-952c42d999c9 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:08.881337 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1241/2458] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:57:08.892577 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1218/2459] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:08 2022] PUT /identity/v3/projects/7372aebdf6554202a45b0bbd7bad0700/users/19c2f15eb1644278b911cc50370da9b7/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:57:08.897278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e902236-699a-44f2-922e-4890e758ae71 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.897690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a16e52-61db-44ba-b434-3125690dff31 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:08.897980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a16e52-61db-44ba-b434-3125690dff31 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:08.898255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a16e52-61db-44ba-b434-3125690dff31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:08.898928 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1e902236-699a-44f2-922e-4890e758ae71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:08.917874 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-e4a16e52-61db-44ba-b434-3125690dff31 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:08.919119 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e4a16e52-61db-44ba-b434-3125690dff31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:08.958017 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e902236-699a-44f2-922e-4890e758ae71 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:08.963590 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:08.982580 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1219/2460] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:57:08.993482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a16e52-61db-44ba-b434-3125690dff31 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:08.995074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e4a16e52-61db-44ba-b434-3125690dff31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:09.010707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bdd52-0ca4-4535-a646-7b6b4d9b4e6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:09.011488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdd52-0ca4-4535-a646-7b6b4d9b4e6a service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:09.012088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdd52-0ca4-4535-a646-7b6b4d9b4e6a service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:09.012088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdd52-0ca4-4535-a646-7b6b4d9b4e6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:09.062204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a16e52-61db-44ba-b434-3125690dff31 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:09.065929 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4e206-4b87-4594-b628-96e7d72be445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:09.066534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4e206-4b87-4594-b628-96e7d72be445 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:09.066840 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4e206-4b87-4594-b628-96e7d72be445 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:09.067160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4e206-4b87-4594-b628-96e7d72be445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:09.077017 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4e206-4b87-4594-b628-96e7d72be445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:09.077341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bdd52-0ca4-4535-a646-7b6b4d9b4e6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:09.078716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4e206-4b87-4594-b628-96e7d72be445 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:09.078870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bdd52-0ca4-4535-a646-7b6b4d9b4e6a service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:09.089270 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1220/2461] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:57:09.090787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1242/2462] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:09.469710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4e206-4b87-4594-b628-96e7d72be445 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:09.483702 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4e206-4b87-4594-b628-96e7d72be445 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:09.487893 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342b497-1739-4e34-91ba-9a7682f57c40 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6af12b3fa05447b88457651a108e93fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ca9b7283d6543faa6a9b1f01fea420b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:09.488534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342b497-1739-4e34-91ba-9a7682f57c40 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:09.488845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342b497-1739-4e34-91ba-9a7682f57c40 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:09.489686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342b497-1739-4e34-91ba-9a7682f57c40 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:09.490392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342b497-1739-4e34-91ba-9a7682f57c40 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:57:09.490727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342b497-1739-4e34-91ba-9a7682f57c40 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:09.492457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342b497-1739-4e34-91ba-9a7682f57c40 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:09.507933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1221/2463] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:57:09.512364 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bdd52-0ca4-4535-a646-7b6b4d9b4e6a service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:09.525105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bdd52-0ca4-4535-a646-7b6b4d9b4e6a service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:09.529796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087595e-5449-4c9f-91bd-74a272b4174b tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6af12b3fa05447b88457651a108e93fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ca9b7283d6543faa6a9b1f01fea420b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:09.530414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087595e-5449-4c9f-91bd-74a272b4174b tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:09.530728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087595e-5449-4c9f-91bd-74a272b4174b tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:09.531028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087595e-5449-4c9f-91bd-74a272b4174b tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:09.531685 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087595e-5449-4c9f-91bd-74a272b4174b tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:09.533721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087595e-5449-4c9f-91bd-74a272b4174b tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:09.567783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1243/2464] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:09 2022] POST /identity/v3/projects => generated 355 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:09.571857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342b497-1739-4e34-91ba-9a7682f57c40 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:09.582271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342b497-1739-4e34-91ba-9a7682f57c40 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:09.589141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:09.644554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f8c3d-f44c-4968-9a2e-4d99fbb15eb7 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6af12b3fa05447b88457651a108e93fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ca9b7283d6543faa6a9b1f01fea420b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:09.645877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f8c3d-f44c-4968-9a2e-4d99fbb15eb7 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:09.646155 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f8c3d-f44c-4968-9a2e-4d99fbb15eb7 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:09.646424 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f8c3d-f44c-4968-9a2e-4d99fbb15eb7 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:09.647221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f8c3d-f44c-4968-9a2e-4d99fbb15eb7 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:57:09.648312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f8c3d-f44c-4968-9a2e-4d99fbb15eb7 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:09.649523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f8c3d-f44c-4968-9a2e-4d99fbb15eb7 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:09.662212 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1222/2465] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:57:09.705507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087595e-5449-4c9f-91bd-74a272b4174b tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:09.718859 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087595e-5449-4c9f-91bd-74a272b4174b tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:09.726659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e06e1b-f702-4961-98d4-ba829887fe74 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6af12b3fa05447b88457651a108e93fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ca9b7283d6543faa6a9b1f01fea420b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:09.727903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e06e1b-f702-4961-98d4-ba829887fe74 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:09.728298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e06e1b-f702-4961-98d4-ba829887fe74 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:09.729224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e06e1b-f702-4961-98d4-ba829887fe74 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] PATH_INFO: `/v3/projects/fda04a22565744678d06c8788859f9f5` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:09.731114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e06e1b-f702-4961-98d4-ba829887fe74 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fda04a22565744678d06c8788859f9f5)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:09.733480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e06e1b-f702-4961-98d4-ba829887fe74 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:09.801245 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d8e06e1b-f702-4961-98d4-ba829887fe74 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fda04a22565744678d06c8788859f9f5', 'request_id': 'req-d8e06e1b-f702-4961-98d4-ba829887fe74'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:09.810164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d8e06e1b-f702-4961-98d4-ba829887fe74 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fda04a22565744678d06c8788859f9f5', 'request_id': 'req-d8e06e1b-f702-4961-98d4-ba829887fe74'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:09.820027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d8e06e1b-f702-4961-98d4-ba829887fe74 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fda04a22565744678d06c8788859f9f5', 'request_id': 'req-d8e06e1b-f702-4961-98d4-ba829887fe74'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:09.837966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d8e06e1b-f702-4961-98d4-ba829887fe74 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] The token cache is being invalidate because project fda04a22565744678d06c8788859f9f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:57:09.838503 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d8e06e1b-f702-4961-98d4-ba829887fe74 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:09.839345 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d8e06e1b-f702-4961-98d4-ba829887fe74 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:09.841577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1244/2466] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:57:09 2022] DELETE /identity/v3/projects/fda04a22565744678d06c8788859f9f5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:10.087863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f8c3d-f44c-4968-9a2e-4d99fbb15eb7 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:10.090950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b93f8c3d-f44c-4968-9a2e-4d99fbb15eb7 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:10.167608 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f8c3d-f44c-4968-9a2e-4d99fbb15eb7 tempest-VolumeQuotaClassesTest-276102780 tempest-VolumeQuotaClassesTest-276102780-project-admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:10.244196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:10.244899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:10.245196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:10.245534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] PATH_INFO: `/v3/users/174b2a2ff4474c2a81209c1a9418f77d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:10.247036 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] RBAC: Authorizing `identity:delete_user(user_id=174b2a2ff4474c2a81209c1a9418f77d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:10.249116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:10.315804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '174b2a2ff4474c2a81209c1a9418f77d'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:10.325809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '174b2a2ff4474c2a81209c1a9418f77d'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:10.326867 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '174b2a2ff4474c2a81209c1a9418f77d'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:10.333985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:10.355716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '174b2a2ff4474c2a81209c1a9418f77d'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:10.368614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '174b2a2ff4474c2a81209c1a9418f77d'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:10.376268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1223/2467] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:57:10 2022] DELETE /identity/v3/users/174b2a2ff4474c2a81209c1a9418f77d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:10.381161 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435e46e-0c35-4909-b295-8ae3ec0edd9c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:10.381615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435e46e-0c35-4909-b295-8ae3ec0edd9c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:10.381958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435e46e-0c35-4909-b295-8ae3ec0edd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:10.408063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-4435e46e-0c35-4909-b295-8ae3ec0edd9c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:10.409473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4435e46e-0c35-4909-b295-8ae3ec0edd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:10.436745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:10.508213 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1245/2468] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:10.519114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:10.522072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:10.596315 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030644ed-b7c7-41c0-b277-33103ea3807e admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:10.600729 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f3031-dc68-4187-a327-06c6ee61bf78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:10.601344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f3031-dc68-4187-a327-06c6ee61bf78 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:10.601618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f3031-dc68-4187-a327-06c6ee61bf78 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:10.601885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f3031-dc68-4187-a327-06c6ee61bf78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:10.611524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f3031-dc68-4187-a327-06c6ee61bf78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:10.612947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f3031-dc68-4187-a327-06c6ee61bf78 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:10.624772 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1224/2469] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:10.982592 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435e46e-0c35-4909-b295-8ae3ec0edd9c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:10.984443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4435e46e-0c35-4909-b295-8ae3ec0edd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:11.000395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435e46e-0c35-4909-b295-8ae3ec0edd9c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:11.008916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:11.010017 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:11.010495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:11.010764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] PATH_INFO: `/v3/projects/572551ada6db42bb868f576d27a5af75` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:11.012700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=572551ada6db42bb868f576d27a5af75)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:11.014845 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:11.079990 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '572551ada6db42bb868f576d27a5af75', 'request_id': 'req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:11.086637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '572551ada6db42bb868f576d27a5af75', 'request_id': 'req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:11.094660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '572551ada6db42bb868f576d27a5af75', 'request_id': 'req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:11.110569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] The token cache is being invalidate because project 572551ada6db42bb868f576d27a5af75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:57:11.110894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:11.111590 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:11.113185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1246/2470] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:57:10 2022] DELETE /identity/v3/projects/572551ada6db42bb868f576d27a5af75 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:11.123244 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f3031-dc68-4187-a327-06c6ee61bf78 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:11.133893 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f3031-dc68-4187-a327-06c6ee61bf78 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:11.205154 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b80e5f-545d-4c9d-aaa6-cea6bf3efd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:11.206056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b80e5f-545d-4c9d-aaa6-cea6bf3efd1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:11.206782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b80e5f-545d-4c9d-aaa6-cea6bf3efd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:11.211186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b80e5f-545d-4c9d-aaa6-cea6bf3efd1e admin admin] PATH_INFO: `/v3/users/6af12b3fa05447b88457651a108e93fb` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:11.213863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b80e5f-545d-4c9d-aaa6-cea6bf3efd1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6af12b3fa05447b88457651a108e93fb)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:11.217204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b80e5f-545d-4c9d-aaa6-cea6bf3efd1e admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:11.279619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b7b80e5f-545d-4c9d-aaa6-cea6bf3efd1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6af12b3fa05447b88457651a108e93fb'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:11.294721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b7b80e5f-545d-4c9d-aaa6-cea6bf3efd1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6af12b3fa05447b88457651a108e93fb'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:11.295243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b7b80e5f-545d-4c9d-aaa6-cea6bf3efd1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6af12b3fa05447b88457651a108e93fb'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:11.309211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b7b80e5f-545d-4c9d-aaa6-cea6bf3efd1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6af12b3fa05447b88457651a108e93fb'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:11.312147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:11.319686 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b7b80e5f-545d-4c9d-aaa6-cea6bf3efd1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6af12b3fa05447b88457651a108e93fb'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:11.331184 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1225/2471] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:57:11 2022] DELETE /identity/v3/users/6af12b3fa05447b88457651a108e93fb => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:11.690469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:11.693611 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:11.727152 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6cc8b-0803-4120-b4e4-60418cf7c7b2 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:12.032767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1b0db-5323-4748-903c-535f124064eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:12.034022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b0db-5323-4748-903c-535f124064eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:12.035532 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b0db-5323-4748-903c-535f124064eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:12.036341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b0db-5323-4748-903c-535f124064eb admin admin] PATH_INFO: `/v3/projects/0ca9b7283d6543faa6a9b1f01fea420b` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:12.042879 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1b0db-5323-4748-903c-535f124064eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ca9b7283d6543faa6a9b1f01fea420b)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:12.045646 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1b0db-5323-4748-903c-535f124064eb admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:12.145880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:12.153496 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-51d1b0db-5323-4748-903c-535f124064eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ca9b7283d6543faa6a9b1f01fea420b', 'request_id': 'req-51d1b0db-5323-4748-903c-535f124064eb'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:12.169724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-51d1b0db-5323-4748-903c-535f124064eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ca9b7283d6543faa6a9b1f01fea420b', 'request_id': 'req-51d1b0db-5323-4748-903c-535f124064eb'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:12.189785 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-51d1b0db-5323-4748-903c-535f124064eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ca9b7283d6543faa6a9b1f01fea420b', 'request_id': 'req-51d1b0db-5323-4748-903c-535f124064eb'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:12.214565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-51d1b0db-5323-4748-903c-535f124064eb admin admin] The token cache is being invalidate because project 0ca9b7283d6543faa6a9b1f01fea420b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:57:12.214866 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-51d1b0db-5323-4748-903c-535f124064eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:12.216544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-51d1b0db-5323-4748-903c-535f124064eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:12.222263 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1247/2472] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:57:11 2022] DELETE /identity/v3/projects/0ca9b7283d6543faa6a9b1f01fea420b => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:12.228857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31452c-58ea-497f-aad8-5bf96c8bbc92 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:12.229798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31452c-58ea-497f-aad8-5bf96c8bbc92 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:12.230287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31452c-58ea-497f-aad8-5bf96c8bbc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:12.268448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-5e31452c-58ea-497f-aad8-5bf96c8bbc92 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:12.269682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5e31452c-58ea-497f-aad8-5bf96c8bbc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:12.415872 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1226/2473] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:12.419418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f585b-44c5-46e4-9019-1929189ca75f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:12.419638 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f585b-44c5-46e4-9019-1929189ca75f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:12.420090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f585b-44c5-46e4-9019-1929189ca75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:12.467326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-675f585b-44c5-46e4-9019-1929189ca75f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:12.468559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-675f585b-44c5-46e4-9019-1929189ca75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:12.496835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1248/2474] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:12.498258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31452c-58ea-497f-aad8-5bf96c8bbc92 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:12.518491 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31452c-58ea-497f-aad8-5bf96c8bbc92 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:12.533351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49776396-197e-4be6-bc0a-e732e9e69f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:12.534012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49776396-197e-4be6-bc0a-e732e9e69f23 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:12.534555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49776396-197e-4be6-bc0a-e732e9e69f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:12.534873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49776396-197e-4be6-bc0a-e732e9e69f23 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:12.536144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49776396-197e-4be6-bc0a-e732e9e69f23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:57:12.536574 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49776396-197e-4be6-bc0a-e732e9e69f23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:12.539087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49776396-197e-4be6-bc0a-e732e9e69f23 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:12.558613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1227/2475] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:57:12.559807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f585b-44c5-46e4-9019-1929189ca75f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:12.572578 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f585b-44c5-46e4-9019-1929189ca75f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:12.577410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2378ea-5cf5-4631-9acf-acb5d587e665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:12.578030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2378ea-5cf5-4631-9acf-acb5d587e665 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:12.578319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2378ea-5cf5-4631-9acf-acb5d587e665 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:12.578563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2378ea-5cf5-4631-9acf-acb5d587e665 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:12.579249 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2378ea-5cf5-4631-9acf-acb5d587e665 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:12.581509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2378ea-5cf5-4631-9acf-acb5d587e665 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:12.628784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1249/2476] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:12 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:12.635146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49776396-197e-4be6-bc0a-e732e9e69f23 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:12.647967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49776396-197e-4be6-bc0a-e732e9e69f23 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:12.722037 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222497d-b006-4367-9d5e-0114bb39e86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:12.723038 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222497d-b006-4367-9d5e-0114bb39e86c admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:12.723038 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222497d-b006-4367-9d5e-0114bb39e86c admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:12.724341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222497d-b006-4367-9d5e-0114bb39e86c admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:12.726110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222497d-b006-4367-9d5e-0114bb39e86c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:12.728890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222497d-b006-4367-9d5e-0114bb39e86c admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:12.757995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1228/2477] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:12 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:12.763081 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2378ea-5cf5-4631-9acf-acb5d587e665 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:12.780956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2378ea-5cf5-4631-9acf-acb5d587e665 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:12.789262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407fe10-d9fa-4a3a-918f-a1719bc73ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:12.790332 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407fe10-d9fa-4a3a-918f-a1719bc73ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:12.790803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407fe10-d9fa-4a3a-918f-a1719bc73ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:12.791374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407fe10-d9fa-4a3a-918f-a1719bc73ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:12.792319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407fe10-d9fa-4a3a-918f-a1719bc73ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:12.794613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407fe10-d9fa-4a3a-918f-a1719bc73ee5 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:12.808916 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1250/2478] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:12.812855 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222497d-b006-4367-9d5e-0114bb39e86c admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:12.823830 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222497d-b006-4367-9d5e-0114bb39e86c admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:12.829643 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f0a0d-7553-4ea8-b4c9-aeffbbf04f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:12.830442 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f0a0d-7553-4ea8-b4c9-aeffbbf04f90 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:12.831089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f0a0d-7553-4ea8-b4c9-aeffbbf04f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:12.831497 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f0a0d-7553-4ea8-b4c9-aeffbbf04f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:12.832957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f0a0d-7553-4ea8-b4c9-aeffbbf04f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:12.834988 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f0a0d-7553-4ea8-b4c9-aeffbbf04f90 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:12.850595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1229/2479] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:12.854835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407fe10-d9fa-4a3a-918f-a1719bc73ee5 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:12.866490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407fe10-d9fa-4a3a-918f-a1719bc73ee5 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:12.872495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137be524-5c13-4755-a6ff-25bb05917b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:12.873596 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137be524-5c13-4755-a6ff-25bb05917b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:12.873936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137be524-5c13-4755-a6ff-25bb05917b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:12.874287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137be524-5c13-4755-a6ff-25bb05917b6f admin admin] PATH_INFO: `/v3/projects/397c74a770624f05a352a7bdf6e9040b/users/8ed484aa9e4c44afad289b485498969b/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:12.898119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137be524-5c13-4755-a6ff-25bb05917b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=397c74a770624f05a352a7bdf6e9040b, user_id=8ed484aa9e4c44afad289b485498969b, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:12.901310 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137be524-5c13-4755-a6ff-25bb05917b6f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:12.916958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1251/2480] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:12 2022] PUT /identity/v3/projects/397c74a770624f05a352a7bdf6e9040b/users/8ed484aa9e4c44afad289b485498969b/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:12.929975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15635bd4-5147-4b6d-843d-56b1497b6acb None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:12.930232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15635bd4-5147-4b6d-843d-56b1497b6acb None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:12.930518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15635bd4-5147-4b6d-843d-56b1497b6acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:12.961557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-15635bd4-5147-4b6d-843d-56b1497b6acb None None] MFA Rules not processed for user `8ed484aa9e4c44afad289b485498969b`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:12.962994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-15635bd4-5147-4b6d-843d-56b1497b6acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:13.020927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:13.065265 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1230/2481] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:13.082020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137be524-5c13-4755-a6ff-25bb05917b6f admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:13.086675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-137be524-5c13-4755-a6ff-25bb05917b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:13.174022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137be524-5c13-4755-a6ff-25bb05917b6f admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:13.180337 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6128f-b8cc-4869-aca2-71ec2b634e81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:13.181303 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6128f-b8cc-4869-aca2-71ec2b634e81 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:13.181687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6128f-b8cc-4869-aca2-71ec2b634e81 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:13.183602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6128f-b8cc-4869-aca2-71ec2b634e81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:13.198166 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd6128f-b8cc-4869-aca2-71ec2b634e81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:13.199893 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd6128f-b8cc-4869-aca2-71ec2b634e81 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:13.213081 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1252/2482] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:14.119976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616046e-6eba-4490-a274-47de4156c61e None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:14.120935 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616046e-6eba-4490-a274-47de4156c61e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:14.121373 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616046e-6eba-4490-a274-47de4156c61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:14.161640 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-3616046e-6eba-4490-a274-47de4156c61e None None] MFA Rules not processed for user `19c2f15eb1644278b911cc50370da9b7`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:14.163274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3616046e-6eba-4490-a274-47de4156c61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:14.269065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1231/2483] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:14.269922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6128f-b8cc-4869-aca2-71ec2b634e81 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:14.271926 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1bd6128f-b8cc-4869-aca2-71ec2b634e81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:14.357602 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6128f-b8cc-4869-aca2-71ec2b634e81 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:14.362647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1fb30-83c8-4a70-8f25-cab5b2d29fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:14.363341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1fb30-83c8-4a70-8f25-cab5b2d29fcc admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:14.363687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1fb30-83c8-4a70-8f25-cab5b2d29fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:14.364048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1fb30-83c8-4a70-8f25-cab5b2d29fcc admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:14.364848 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1fb30-83c8-4a70-8f25-cab5b2d29fcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:14.366774 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1fb30-83c8-4a70-8f25-cab5b2d29fcc admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:14.389515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:14.405095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1253/2484] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:14 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:14.406334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616046e-6eba-4490-a274-47de4156c61e None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:14.419840 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616046e-6eba-4490-a274-47de4156c61e None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:14.482890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184164f2-ff0e-4a78-aa25-5456947967cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:14.483576 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184164f2-ff0e-4a78-aa25-5456947967cc admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:14.483912 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184164f2-ff0e-4a78-aa25-5456947967cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:14.484241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184164f2-ff0e-4a78-aa25-5456947967cc admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:14.484956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184164f2-ff0e-4a78-aa25-5456947967cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:14.487272 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184164f2-ff0e-4a78-aa25-5456947967cc admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:14.516303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1232/2485] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:14 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:14.521132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1fb30-83c8-4a70-8f25-cab5b2d29fcc admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:14.532329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1fb30-83c8-4a70-8f25-cab5b2d29fcc admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:14.537900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a36dc8-d547-4464-b7a0-793a9acf435e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:14.538514 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a36dc8-d547-4464-b7a0-793a9acf435e admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:14.538820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a36dc8-d547-4464-b7a0-793a9acf435e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:14.539095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a36dc8-d547-4464-b7a0-793a9acf435e admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:14.539821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a36dc8-d547-4464-b7a0-793a9acf435e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:14.541400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a36dc8-d547-4464-b7a0-793a9acf435e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:14.556498 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1254/2486] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:14.557768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184164f2-ff0e-4a78-aa25-5456947967cc admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:14.569964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184164f2-ff0e-4a78-aa25-5456947967cc admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:14.575466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013594bc-817f-4ce4-9153-3833002108cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:14.576137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013594bc-817f-4ce4-9153-3833002108cc admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:14.576415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013594bc-817f-4ce4-9153-3833002108cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:14.576689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013594bc-817f-4ce4-9153-3833002108cc admin admin] PATH_INFO: `/v3/projects/841a6466aef743a8b19c027d3c964e7b/users/d60f1886601b4e5d911146fb371f28b7/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:14.591831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013594bc-817f-4ce4-9153-3833002108cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=841a6466aef743a8b19c027d3c964e7b, user_id=d60f1886601b4e5d911146fb371f28b7, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:14.594479 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013594bc-817f-4ce4-9153-3833002108cc admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:14.607293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1233/2487] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:14 2022] PUT /identity/v3/projects/841a6466aef743a8b19c027d3c964e7b/users/d60f1886601b4e5d911146fb371f28b7/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:14.611262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a36dc8-d547-4464-b7a0-793a9acf435e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:14.622545 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a36dc8-d547-4464-b7a0-793a9acf435e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:14.697620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c2107-375e-49d0-b5d6-4b128db1d2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:14.699407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c2107-375e-49d0-b5d6-4b128db1d2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:14.700205 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c2107-375e-49d0-b5d6-4b128db1d2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:14.700755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c2107-375e-49d0-b5d6-4b128db1d2a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:14.702516 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c2107-375e-49d0-b5d6-4b128db1d2a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:14.704404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c2107-375e-49d0-b5d6-4b128db1d2a9 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:14.728207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1255/2488] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:14.729740 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013594bc-817f-4ce4-9153-3833002108cc admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:14.747154 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013594bc-817f-4ce4-9153-3833002108cc admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:14.754730 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25405763-44a1-42ae-a1d6-30f86eee3db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:14.756161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25405763-44a1-42ae-a1d6-30f86eee3db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:14.756461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25405763-44a1-42ae-a1d6-30f86eee3db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:14.756744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25405763-44a1-42ae-a1d6-30f86eee3db7 admin admin] PATH_INFO: `/v3/projects/841a6466aef743a8b19c027d3c964e7b/users/d60f1886601b4e5d911146fb371f28b7/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:14.759034 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25405763-44a1-42ae-a1d6-30f86eee3db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=841a6466aef743a8b19c027d3c964e7b, user_id=d60f1886601b4e5d911146fb371f28b7, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:14.761878 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25405763-44a1-42ae-a1d6-30f86eee3db7 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:14.775351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1234/2489] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:14 2022] PUT /identity/v3/projects/841a6466aef743a8b19c027d3c964e7b/users/d60f1886601b4e5d911146fb371f28b7/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:20.087761 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc418dd-718b-413e-aaf6-b214492078c5 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:20.089338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc418dd-718b-413e-aaf6-b214492078c5 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:20.089775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc418dd-718b-413e-aaf6-b214492078c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:20.141987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-8bc418dd-718b-413e-aaf6-b214492078c5 None None] MFA Rules not processed for user `d60f1886601b4e5d911146fb371f28b7`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:20.146331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8bc418dd-718b-413e-aaf6-b214492078c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:20.237900 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1256/2490] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:20.253404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25405763-44a1-42ae-a1d6-30f86eee3db7 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:20.256430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-25405763-44a1-42ae-a1d6-30f86eee3db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:20.294075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:20.343628 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25405763-44a1-42ae-a1d6-30f86eee3db7 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:20.351403 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b83593-4a03-4027-9714-dd4391d6ecd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:20.352158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b83593-4a03-4027-9714-dd4391d6ecd0 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:20.352544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b83593-4a03-4027-9714-dd4391d6ecd0 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:20.352902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b83593-4a03-4027-9714-dd4391d6ecd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:20.437318 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b83593-4a03-4027-9714-dd4391d6ecd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:20.439043 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b83593-4a03-4027-9714-dd4391d6ecd0 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:20.450398 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1235/2491] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:57:21.592442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc418dd-718b-413e-aaf6-b214492078c5 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:21.680176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc418dd-718b-413e-aaf6-b214492078c5 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:21.685513 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:21.686227 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:21.686505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:21.686832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] PATH_INFO: `/v3/users/8ed484aa9e4c44afad289b485498969b` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:21.687993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ed484aa9e4c44afad289b485498969b)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:21.690780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:21.703934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:21.748915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ed484aa9e4c44afad289b485498969b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:21.764840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ed484aa9e4c44afad289b485498969b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:21.774777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ed484aa9e4c44afad289b485498969b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:21.777980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ed484aa9e4c44afad289b485498969b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:21.778822 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ed484aa9e4c44afad289b485498969b'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:21.800457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1257/2492] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:57:21 2022] DELETE /identity/v3/users/8ed484aa9e4c44afad289b485498969b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:21.804256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89f2d9-e386-4c76-9068-aab7d7a6c80f None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:21.804776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89f2d9-e386-4c76-9068-aab7d7a6c80f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:21.805074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89f2d9-e386-4c76-9068-aab7d7a6c80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:21.829972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-3d89f2d9-e386-4c76-9068-aab7d7a6c80f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:21.831452 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3d89f2d9-e386-4c76-9068-aab7d7a6c80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:21.916811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1236/2493] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:57:21.929520 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:21.931755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:22.041889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9bcc7-d3dc-42a2-8ce6-03e104cace34 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:22.049754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6caf7be-87d3-4609-abce-67f38848bc57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:22.050752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caf7be-87d3-4609-abce-67f38848bc57 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:22.051082 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caf7be-87d3-4609-abce-67f38848bc57 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:22.051968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caf7be-87d3-4609-abce-67f38848bc57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:22.078150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6caf7be-87d3-4609-abce-67f38848bc57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:22.081309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6caf7be-87d3-4609-abce-67f38848bc57 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:22.115507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1258/2494] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:57:22.539847 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89f2d9-e386-4c76-9068-aab7d7a6c80f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:22.541775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3d89f2d9-e386-4c76-9068-aab7d7a6c80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:22.568353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89f2d9-e386-4c76-9068-aab7d7a6c80f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:22.585565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c30736-ae4f-44ef-aff7-d68accbf59d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:22.586598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c30736-ae4f-44ef-aff7-d68accbf59d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:22.586985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c30736-ae4f-44ef-aff7-d68accbf59d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:22.589288 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c30736-ae4f-44ef-aff7-d68accbf59d6 admin admin] PATH_INFO: `/v3/projects/397c74a770624f05a352a7bdf6e9040b` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:22.591572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c30736-ae4f-44ef-aff7-d68accbf59d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=397c74a770624f05a352a7bdf6e9040b)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:22.597114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c30736-ae4f-44ef-aff7-d68accbf59d6 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:22.694906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b0c30736-ae4f-44ef-aff7-d68accbf59d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '397c74a770624f05a352a7bdf6e9040b', 'request_id': 'req-b0c30736-ae4f-44ef-aff7-d68accbf59d6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:22.702874 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b0c30736-ae4f-44ef-aff7-d68accbf59d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '397c74a770624f05a352a7bdf6e9040b', 'request_id': 'req-b0c30736-ae4f-44ef-aff7-d68accbf59d6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:22.717028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b0c30736-ae4f-44ef-aff7-d68accbf59d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '397c74a770624f05a352a7bdf6e9040b', 'request_id': 'req-b0c30736-ae4f-44ef-aff7-d68accbf59d6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:22.718687 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:22.733730 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b0c30736-ae4f-44ef-aff7-d68accbf59d6 admin admin] The token cache is being invalidate because project 397c74a770624f05a352a7bdf6e9040b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:57:22.734038 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b0c30736-ae4f-44ef-aff7-d68accbf59d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:22.735024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b0c30736-ae4f-44ef-aff7-d68accbf59d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:22.742069 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1237/2495] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:57:22 2022] DELETE /identity/v3/projects/397c74a770624f05a352a7bdf6e9040b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:22.751067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f1269-82fd-42bc-a83c-eb0e6ee818f7 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:22.752359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f1269-82fd-42bc-a83c-eb0e6ee818f7 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:22.753030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f1269-82fd-42bc-a83c-eb0e6ee818f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:22.783452 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-a93f1269-82fd-42bc-a83c-eb0e6ee818f7 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:22.785039 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a93f1269-82fd-42bc-a83c-eb0e6ee818f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:22.878191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1259/2496] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:22.879353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb8e58-953f-47c2-9cc0-9b3659db42d2 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:22.882103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb8e58-953f-47c2-9cc0-9b3659db42d2 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:22.883089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb8e58-953f-47c2-9cc0-9b3659db42d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:22.920285 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-74eb8e58-953f-47c2-9cc0-9b3659db42d2 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:22.922216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-74eb8e58-953f-47c2-9cc0-9b3659db42d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:22.936964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1238/2497] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:22.941523 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f1269-82fd-42bc-a83c-eb0e6ee818f7 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:22.957434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f1269-82fd-42bc-a83c-eb0e6ee818f7 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:22.964895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15bb5e-9a7e-43c1-934c-9cc5ce929430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:22.966390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15bb5e-9a7e-43c1-934c-9cc5ce929430 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:22.966911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15bb5e-9a7e-43c1-934c-9cc5ce929430 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:22.967359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15bb5e-9a7e-43c1-934c-9cc5ce929430 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:22.969058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15bb5e-9a7e-43c1-934c-9cc5ce929430 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:57:22.969548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15bb5e-9a7e-43c1-934c-9cc5ce929430 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:22.971577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15bb5e-9a7e-43c1-934c-9cc5ce929430 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:22.993810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1260/2498] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:57:22.996239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb8e58-953f-47c2-9cc0-9b3659db42d2 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:23.012283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb8e58-953f-47c2-9cc0-9b3659db42d2 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:23.018914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15827f7d-293d-42c6-863a-db7fad30aa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:23.019653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15827f7d-293d-42c6-863a-db7fad30aa9e admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:23.020272 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15827f7d-293d-42c6-863a-db7fad30aa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:23.021381 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15827f7d-293d-42c6-863a-db7fad30aa9e admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:23.022457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15827f7d-293d-42c6-863a-db7fad30aa9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:23.025949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15827f7d-293d-42c6-863a-db7fad30aa9e admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:23.068828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15bb5e-9a7e-43c1-934c-9cc5ce929430 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:23.076222 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1239/2499] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:22 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:23.080684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15bb5e-9a7e-43c1-934c-9cc5ce929430 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:23.117757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:23.224983 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e7a2c-d39b-4d1e-9945-cd2ddf972905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:23.225614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e7a2c-d39b-4d1e-9945-cd2ddf972905 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:23.225891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e7a2c-d39b-4d1e-9945-cd2ddf972905 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:23.226162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e7a2c-d39b-4d1e-9945-cd2ddf972905 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:23.226865 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e7a2c-d39b-4d1e-9945-cd2ddf972905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:23.229284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e7a2c-d39b-4d1e-9945-cd2ddf972905 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:23.266417 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1261/2500] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:23 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:23.267982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15827f7d-293d-42c6-863a-db7fad30aa9e admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:23.280769 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15827f7d-293d-42c6-863a-db7fad30aa9e admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:23.287129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788793de-9370-4193-ab5f-91f350d78231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:23.288089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788793de-9370-4193-ab5f-91f350d78231 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:23.288681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788793de-9370-4193-ab5f-91f350d78231 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:23.289140 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788793de-9370-4193-ab5f-91f350d78231 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:23.290459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788793de-9370-4193-ab5f-91f350d78231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:23.292651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788793de-9370-4193-ab5f-91f350d78231 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:23.304579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1240/2501] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:23.308690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e7a2c-d39b-4d1e-9945-cd2ddf972905 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:23.329999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e7a2c-d39b-4d1e-9945-cd2ddf972905 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:23.337074 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027f97b-7f1f-4d0f-893d-c0ee3bbe27ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:23.338134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027f97b-7f1f-4d0f-893d-c0ee3bbe27ea admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:23.338720 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027f97b-7f1f-4d0f-893d-c0ee3bbe27ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:23.339159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027f97b-7f1f-4d0f-893d-c0ee3bbe27ea admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:23.340159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027f97b-7f1f-4d0f-893d-c0ee3bbe27ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:23.342001 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027f97b-7f1f-4d0f-893d-c0ee3bbe27ea admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:23.378565 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1262/2502] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:23.381335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788793de-9370-4193-ab5f-91f350d78231 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:23.403346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788793de-9370-4193-ab5f-91f350d78231 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:23.410176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd809d53-de2d-43fe-9335-0b0b609a7bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:23.412013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd809d53-de2d-43fe-9335-0b0b609a7bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:23.412775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd809d53-de2d-43fe-9335-0b0b609a7bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:23.413217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd809d53-de2d-43fe-9335-0b0b609a7bfc admin admin] PATH_INFO: `/v3/projects/ad10a9661c824531b53ebf676390432b/users/63d148ece7534a948d66d82e27435e68/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:23.439397 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd809d53-de2d-43fe-9335-0b0b609a7bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad10a9661c824531b53ebf676390432b, user_id=63d148ece7534a948d66d82e27435e68, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:23.442303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd809d53-de2d-43fe-9335-0b0b609a7bfc admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:23.472887 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1241/2503] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:23 2022] PUT /identity/v3/projects/ad10a9661c824531b53ebf676390432b/users/63d148ece7534a948d66d82e27435e68/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:23.475700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665fb5f-ec5a-40d5-ae04-794060733d0c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:23.476008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665fb5f-ec5a-40d5-ae04-794060733d0c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:23.476300 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665fb5f-ec5a-40d5-ae04-794060733d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:23.530183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-0665fb5f-ec5a-40d5-ae04-794060733d0c None None] MFA Rules not processed for user `63d148ece7534a948d66d82e27435e68`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:23.534189 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-0665fb5f-ec5a-40d5-ae04-794060733d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:23.636264 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1263/2504] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:23.654151 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd809d53-de2d-43fe-9335-0b0b609a7bfc admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:23.658483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-bd809d53-de2d-43fe-9335-0b0b609a7bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:23.747488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd809d53-de2d-43fe-9335-0b0b609a7bfc admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:23.752616 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01fdde-22f3-4abd-b772-cd8546b99b65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:23.753178 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01fdde-22f3-4abd-b772-cd8546b99b65 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:23.753458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01fdde-22f3-4abd-b772-cd8546b99b65 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:23.753731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01fdde-22f3-4abd-b772-cd8546b99b65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:23.763809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01fdde-22f3-4abd-b772-cd8546b99b65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:23.765258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01fdde-22f3-4abd-b772-cd8546b99b65 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:23.775973 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1242/2505] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:26.469896 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665fb5f-ec5a-40d5-ae04-794060733d0c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:26.541437 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665fb5f-ec5a-40d5-ae04-794060733d0c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:26.547766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:26.548608 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:26.548938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:26.549251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] PATH_INFO: `/v3/users/63d148ece7534a948d66d82e27435e68` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:26.550546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63d148ece7534a948d66d82e27435e68)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:26.552958 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:26.587644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:26.608763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63d148ece7534a948d66d82e27435e68'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:26.621925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63d148ece7534a948d66d82e27435e68'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:26.631002 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63d148ece7534a948d66d82e27435e68'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:26.632430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63d148ece7534a948d66d82e27435e68'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:26.633061 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63d148ece7534a948d66d82e27435e68'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:26.646676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1264/2506] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:57:26 2022] DELETE /identity/v3/users/63d148ece7534a948d66d82e27435e68 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:26.652444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fbda3-c4c4-497b-811c-53ee6519f1a2 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:26.652974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fbda3-c4c4-497b-811c-53ee6519f1a2 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:26.653275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9fbda3-c4c4-497b-811c-53ee6519f1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:26.675331 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-2f9fbda3-c4c4-497b-811c-53ee6519f1a2 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:26.676619 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2f9fbda3-c4c4-497b-811c-53ee6519f1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:26.739439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1243/2507] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:26.750091 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:26.752315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:26.814856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0383e43-cf07-432a-b537-6e1eb709bf58 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:26.819274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96797d4-c17e-4f2d-b522-ae8d9bc28439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:26.819929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96797d4-c17e-4f2d-b522-ae8d9bc28439 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:26.820220 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96797d4-c17e-4f2d-b522-ae8d9bc28439 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:26.820498 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96797d4-c17e-4f2d-b522-ae8d9bc28439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:26.829787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96797d4-c17e-4f2d-b522-ae8d9bc28439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:26.831289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96797d4-c17e-4f2d-b522-ae8d9bc28439 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:26.842141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1265/2508] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:27.174159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fbda3-c4c4-497b-811c-53ee6519f1a2 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:27.175769 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2f9fbda3-c4c4-497b-811c-53ee6519f1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:27.195742 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9fbda3-c4c4-497b-811c-53ee6519f1a2 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:27.203509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c86da-9c09-471e-8c74-369038e7ffd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:27.204517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c86da-9c09-471e-8c74-369038e7ffd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:27.204787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c86da-9c09-471e-8c74-369038e7ffd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:27.205274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c86da-9c09-471e-8c74-369038e7ffd8 admin admin] PATH_INFO: `/v3/projects/ad10a9661c824531b53ebf676390432b` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:27.206998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c86da-9c09-471e-8c74-369038e7ffd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad10a9661c824531b53ebf676390432b)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:27.209786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c86da-9c09-471e-8c74-369038e7ffd8 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:27.235325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:27.261060 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-845c86da-9c09-471e-8c74-369038e7ffd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad10a9661c824531b53ebf676390432b', 'request_id': 'req-845c86da-9c09-471e-8c74-369038e7ffd8'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:27.267135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-845c86da-9c09-471e-8c74-369038e7ffd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad10a9661c824531b53ebf676390432b', 'request_id': 'req-845c86da-9c09-471e-8c74-369038e7ffd8'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:27.275062 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-845c86da-9c09-471e-8c74-369038e7ffd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad10a9661c824531b53ebf676390432b', 'request_id': 'req-845c86da-9c09-471e-8c74-369038e7ffd8'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:27.286447 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-845c86da-9c09-471e-8c74-369038e7ffd8 admin admin] The token cache is being invalidate because project ad10a9661c824531b53ebf676390432b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:57:27.286760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-845c86da-9c09-471e-8c74-369038e7ffd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:27.287405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-845c86da-9c09-471e-8c74-369038e7ffd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:27.288964 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1244/2509] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:57:27 2022] DELETE /identity/v3/projects/ad10a9661c824531b53ebf676390432b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:27.294030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d3ff9-184d-423f-8a7f-09b4081df47d None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:27.294349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d3ff9-184d-423f-8a7f-09b4081df47d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:27.294675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d3ff9-184d-423f-8a7f-09b4081df47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:27.316282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-b73d3ff9-184d-423f-8a7f-09b4081df47d None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:27.318141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b73d3ff9-184d-423f-8a7f-09b4081df47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:27.382963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1266/2510] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:27.392040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296596a5-c1d1-4a24-b730-7116ba9ecd67 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:27.392316 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296596a5-c1d1-4a24-b730-7116ba9ecd67 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:27.392584 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296596a5-c1d1-4a24-b730-7116ba9ecd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:27.417051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-296596a5-c1d1-4a24-b730-7116ba9ecd67 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:27.418385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-296596a5-c1d1-4a24-b730-7116ba9ecd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:27.432538 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1245/2511] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:27.437463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d3ff9-184d-423f-8a7f-09b4081df47d None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:27.448837 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d3ff9-184d-423f-8a7f-09b4081df47d None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:27.454021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a5b43-acd7-47a3-af06-21fabdce6a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:27.454963 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a5b43-acd7-47a3-af06-21fabdce6a78 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:27.455343 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a5b43-acd7-47a3-af06-21fabdce6a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:27.455713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a5b43-acd7-47a3-af06-21fabdce6a78 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:27.456788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a5b43-acd7-47a3-af06-21fabdce6a78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:57:27.457233 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a5b43-acd7-47a3-af06-21fabdce6a78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:27.459629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a5b43-acd7-47a3-af06-21fabdce6a78 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:27.478666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1267/2512] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:57:27.489081 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296596a5-c1d1-4a24-b730-7116ba9ecd67 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:27.503097 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296596a5-c1d1-4a24-b730-7116ba9ecd67 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:27.508402 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe9161-cecd-4564-be4c-b23537a0386f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:27.509863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe9161-cecd-4564-be4c-b23537a0386f admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:27.509863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe9161-cecd-4564-be4c-b23537a0386f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:27.510195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe9161-cecd-4564-be4c-b23537a0386f admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:27.510665 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe9161-cecd-4564-be4c-b23537a0386f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:27.512734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe9161-cecd-4564-be4c-b23537a0386f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:27.547100 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1246/2513] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:27 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:27.551284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a5b43-acd7-47a3-af06-21fabdce6a78 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:27.562517 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a5b43-acd7-47a3-af06-21fabdce6a78 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:27.597907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:27.625460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8ba19-e705-4520-99db-73f9699a74ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:27.626072 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8ba19-e705-4520-99db-73f9699a74ca admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:27.626348 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8ba19-e705-4520-99db-73f9699a74ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:27.626613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8ba19-e705-4520-99db-73f9699a74ca admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:27.627293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8ba19-e705-4520-99db-73f9699a74ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:27.629200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8ba19-e705-4520-99db-73f9699a74ca admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:27.653058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1268/2514] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:27 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:27.657406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe9161-cecd-4564-be4c-b23537a0386f admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:27.668144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe9161-cecd-4564-be4c-b23537a0386f admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:27.672749 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925db7e-c09c-4e6e-9a24-8efb718cbb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:27.673328 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925db7e-c09c-4e6e-9a24-8efb718cbb1e admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:27.673603 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925db7e-c09c-4e6e-9a24-8efb718cbb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:27.673887 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925db7e-c09c-4e6e-9a24-8efb718cbb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:27.674552 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925db7e-c09c-4e6e-9a24-8efb718cbb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:27.676197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d925db7e-c09c-4e6e-9a24-8efb718cbb1e admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:27.686897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1247/2515] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:27.691630 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8ba19-e705-4520-99db-73f9699a74ca admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:27.702818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8ba19-e705-4520-99db-73f9699a74ca admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:27.707414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fc8b4-b88c-4f4b-a2e4-860856714cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:27.708858 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fc8b4-b88c-4f4b-a2e4-860856714cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:27.709131 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fc8b4-b88c-4f4b-a2e4-860856714cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:27.709404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fc8b4-b88c-4f4b-a2e4-860856714cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:27.710086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fc8b4-b88c-4f4b-a2e4-860856714cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:27.711710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fc8b4-b88c-4f4b-a2e4-860856714cc6 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:27.721775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1269/2516] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:27.726307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925db7e-c09c-4e6e-9a24-8efb718cbb1e admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:27.737161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925db7e-c09c-4e6e-9a24-8efb718cbb1e admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:27.742129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326336d6-f4e1-4e9e-8354-87f55d011b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:27.742833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326336d6-f4e1-4e9e-8354-87f55d011b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:27.743113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326336d6-f4e1-4e9e-8354-87f55d011b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:27.743388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326336d6-f4e1-4e9e-8354-87f55d011b93 admin admin] PATH_INFO: `/v3/projects/47ea1b6ed7d647b8987e6c128248cab6/users/802ed5c51d184f6c925f31ab6020ac93/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:27.758857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326336d6-f4e1-4e9e-8354-87f55d011b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ea1b6ed7d647b8987e6c128248cab6, user_id=802ed5c51d184f6c925f31ab6020ac93, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:27.761492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326336d6-f4e1-4e9e-8354-87f55d011b93 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:27.774842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1248/2517] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:27 2022] PUT /identity/v3/projects/47ea1b6ed7d647b8987e6c128248cab6/users/802ed5c51d184f6c925f31ab6020ac93/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:27.780769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94602886-3735-4e78-a844-6d40a9a6683c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:27.781077 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94602886-3735-4e78-a844-6d40a9a6683c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:27.781353 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94602886-3735-4e78-a844-6d40a9a6683c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:27.801192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-94602886-3735-4e78-a844-6d40a9a6683c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:27.803308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-94602886-3735-4e78-a844-6d40a9a6683c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:27.864931 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1270/2518] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:27.874953 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326336d6-f4e1-4e9e-8354-87f55d011b93 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:27.876723 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-326336d6-f4e1-4e9e-8354-87f55d011b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:27.941682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326336d6-f4e1-4e9e-8354-87f55d011b93 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:27.946476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c474072-5102-49c0-83db-251aeef5ed26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:27.947312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c474072-5102-49c0-83db-251aeef5ed26 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:27.947566 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c474072-5102-49c0-83db-251aeef5ed26 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:27.947925 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c474072-5102-49c0-83db-251aeef5ed26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:27.958267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c474072-5102-49c0-83db-251aeef5ed26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:27.959757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c474072-5102-49c0-83db-251aeef5ed26 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:27.970122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1249/2519] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:57:28.364342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94602886-3735-4e78-a844-6d40a9a6683c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:28.374831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94602886-3735-4e78-a844-6d40a9a6683c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:28.378862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac5202-67f3-47da-999e-a685a3120d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:28.379423 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac5202-67f3-47da-999e-a685a3120d8d service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:28.380362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac5202-67f3-47da-999e-a685a3120d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:28.380809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac5202-67f3-47da-999e-a685a3120d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:28.383207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-3eac5202-67f3-47da-999e-a685a3120d8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:28.401566 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac5202-67f3-47da-999e-a685a3120d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:28.403472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac5202-67f3-47da-999e-a685a3120d8d service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:28.414730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1271/2520] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:34.166139 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c474072-5102-49c0-83db-251aeef5ed26 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:34.168874 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3c474072-5102-49c0-83db-251aeef5ed26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:34.212302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c474072-5102-49c0-83db-251aeef5ed26 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:34.222942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:34.223876 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:34.224480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:34.224610 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] PATH_INFO: `/v3/users/a357a3d27e4e41adae1c56ee39ae5aa7` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:34.226022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a357a3d27e4e41adae1c56ee39ae5aa7)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:34.229095 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:34.312179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb5ec3-9d6b-4e8e-b5cf-9c8aabd1c7ea None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:34.312877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb5ec3-9d6b-4e8e-b5cf-9c8aabd1c7ea None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:34.312877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb5ec3-9d6b-4e8e-b5cf-9c8aabd1c7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:34.336250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:34.339176 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a357a3d27e4e41adae1c56ee39ae5aa7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:34.354393 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-1ecb5ec3-9d6b-4e8e-b5cf-9c8aabd1c7ea None None] MFA Rules not processed for user `802ed5c51d184f6c925f31ab6020ac93`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:34.355773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1ecb5ec3-9d6b-4e8e-b5cf-9c8aabd1c7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:34.356022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a357a3d27e4e41adae1c56ee39ae5aa7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:34.356720 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a357a3d27e4e41adae1c56ee39ae5aa7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:34.372334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a357a3d27e4e41adae1c56ee39ae5aa7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:34.382998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a357a3d27e4e41adae1c56ee39ae5aa7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:34.393565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1250/2521] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:57:34 2022] DELETE /identity/v3/users/a357a3d27e4e41adae1c56ee39ae5aa7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:34.445847 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1272/2522] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:34.450522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:34.453771 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:34.574278 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45c325-9491-48e1-8467-04ec0e1ddc00 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:34.582799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c1245-391e-4658-9236-cbf01db98022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:34.586943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c1245-391e-4658-9236-cbf01db98022 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:34.587227 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c1245-391e-4658-9236-cbf01db98022 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:34.587518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c1245-391e-4658-9236-cbf01db98022 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:34.588223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c1245-391e-4658-9236-cbf01db98022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:34.590173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c1245-391e-4658-9236-cbf01db98022 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:34.621675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1251/2523] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:34 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:34.626591 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb5ec3-9d6b-4e8e-b5cf-9c8aabd1c7ea None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:34.633165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c1245-391e-4658-9236-cbf01db98022 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:34.635980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b55c1245-391e-4658-9236-cbf01db98022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:34.640860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb5ec3-9d6b-4e8e-b5cf-9c8aabd1c7ea None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:34.685515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:34.721640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9022c-43d3-4c0d-bb05-cb35368daf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:34.722505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9022c-43d3-4c0d-bb05-cb35368daf5d admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:34.722852 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9022c-43d3-4c0d-bb05-cb35368daf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:34.723101 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9022c-43d3-4c0d-bb05-cb35368daf5d admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:34.724000 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9022c-43d3-4c0d-bb05-cb35368daf5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:34.726034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9022c-43d3-4c0d-bb05-cb35368daf5d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:34.738927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c1245-391e-4658-9236-cbf01db98022 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:34.746796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:34.747918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:34.748239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:34.748544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a admin admin] PATH_INFO: `/v3/projects/7f9fbd8d38b743e9915a370d8f1d77c4` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:34.749811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f9fbd8d38b743e9915a370d8f1d77c4)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:34.751774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:34.753246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1273/2524] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:34 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:34.757709 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9022c-43d3-4c0d-bb05-cb35368daf5d admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:34.773312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9022c-43d3-4c0d-bb05-cb35368daf5d admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:34.779247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e003c3-65c9-497f-92aa-f095281bcce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:34.779807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e003c3-65c9-497f-92aa-f095281bcce0 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:34.780086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e003c3-65c9-497f-92aa-f095281bcce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:34.780357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e003c3-65c9-497f-92aa-f095281bcce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:34.781064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e003c3-65c9-497f-92aa-f095281bcce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:34.783380 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e003c3-65c9-497f-92aa-f095281bcce0 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:34.798512 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1274/2525] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:34.799934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e003c3-65c9-497f-92aa-f095281bcce0 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:34.813190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e003c3-65c9-497f-92aa-f095281bcce0 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:34.813436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f9fbd8d38b743e9915a370d8f1d77c4', 'request_id': 'req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:34.815592 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:34.821842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f9fbd8d38b743e9915a370d8f1d77c4', 'request_id': 'req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:34.837462 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f9fbd8d38b743e9915a370d8f1d77c4', 'request_id': 'req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:34.851142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a admin admin] The token cache is being invalidate because project 7f9fbd8d38b743e9915a370d8f1d77c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:57:34.851529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:34.852586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bbbb0cca-0e66-4203-a50d-28d7962cfc8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:34.855005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1252/2526] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:57:34 2022] DELETE /identity/v3/projects/7f9fbd8d38b743e9915a370d8f1d77c4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:57:34.859801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ae82f-10f8-46f4-9ddd-6626bdd95310 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:34.860165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ae82f-10f8-46f4-9ddd-6626bdd95310 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:34.860510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ae82f-10f8-46f4-9ddd-6626bdd95310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:34.883138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cadc33-06f6-4a62-a1e5-7b322ff0657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:34.883821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cadc33-06f6-4a62-a1e5-7b322ff0657d admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:34.884198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cadc33-06f6-4a62-a1e5-7b322ff0657d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:34.884498 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cadc33-06f6-4a62-a1e5-7b322ff0657d admin admin] PATH_INFO: `/v3/projects/a05914c4ddfa4c72b485a28548b9d83e/users/6c2457e56a1841c0907777cbf6284921/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:34.889487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-d01ae82f-10f8-46f4-9ddd-6626bdd95310 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:34.890995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d01ae82f-10f8-46f4-9ddd-6626bdd95310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:34.906417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1253/2527] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:57:34.906660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cadc33-06f6-4a62-a1e5-7b322ff0657d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a05914c4ddfa4c72b485a28548b9d83e, user_id=6c2457e56a1841c0907777cbf6284921, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:34.906660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cadc33-06f6-4a62-a1e5-7b322ff0657d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:34.908331 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd1f31-baf0-468a-a551-549667fc5057 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:34.908689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd1f31-baf0-468a-a551-549667fc5057 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:34.909056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd1f31-baf0-468a-a551-549667fc5057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:34.922058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1275/2528] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:34 2022] PUT /identity/v3/projects/a05914c4ddfa4c72b485a28548b9d83e/users/6c2457e56a1841c0907777cbf6284921/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:57:34.926392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cadc33-06f6-4a62-a1e5-7b322ff0657d admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:34.929298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-91cadc33-06f6-4a62-a1e5-7b322ff0657d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:34.932811 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c4bd1f31-baf0-468a-a551-549667fc5057 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:34.934032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c4bd1f31-baf0-468a-a551-549667fc5057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:35.002816 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1254/2529] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:35.005595 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cadc33-06f6-4a62-a1e5-7b322ff0657d admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:35.007208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd1f31-baf0-468a-a551-549667fc5057 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:35.011630 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86157e7-d7c9-47e0-ab72-6565ba3b3093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:35.012262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86157e7-d7c9-47e0-ab72-6565ba3b3093 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:35.012541 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86157e7-d7c9-47e0-ab72-6565ba3b3093 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:35.012816 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86157e7-d7c9-47e0-ab72-6565ba3b3093 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:35.013518 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86157e7-d7c9-47e0-ab72-6565ba3b3093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:35.015207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86157e7-d7c9-47e0-ab72-6565ba3b3093 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:35.017291 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd1f31-baf0-468a-a551-549667fc5057 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:35.022468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd45ef4-af2d-4594-a9c4-ac909e1962a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:35.023243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd45ef4-af2d-4594-a9c4-ac909e1962a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:35.023650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd45ef4-af2d-4594-a9c4-ac909e1962a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:35.024141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd45ef4-af2d-4594-a9c4-ac909e1962a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:35.025099 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd45ef4-af2d-4594-a9c4-ac909e1962a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:57:35.025513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd45ef4-af2d-4594-a9c4-ac909e1962a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:35.026670 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1276/2530] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:57:35.030028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd45ef4-af2d-4594-a9c4-ac909e1962a1 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:35.031500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86157e7-d7c9-47e0-ab72-6565ba3b3093 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:35.042019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1255/2531] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:57:35.042577 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86157e7-d7c9-47e0-ab72-6565ba3b3093 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:35.046341 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd45ef4-af2d-4594-a9c4-ac909e1962a1 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:35.052668 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200441d5-86f2-46ee-8843-24622e1c70bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:35.053808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200441d5-86f2-46ee-8843-24622e1c70bb admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:35.054064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200441d5-86f2-46ee-8843-24622e1c70bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:35.054354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200441d5-86f2-46ee-8843-24622e1c70bb admin admin] PATH_INFO: `/v3/projects/a05914c4ddfa4c72b485a28548b9d83e/users/6c2457e56a1841c0907777cbf6284921/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:35.056618 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200441d5-86f2-46ee-8843-24622e1c70bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a05914c4ddfa4c72b485a28548b9d83e, user_id=6c2457e56a1841c0907777cbf6284921, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:35.058868 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd45ef4-af2d-4594-a9c4-ac909e1962a1 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:35.059334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200441d5-86f2-46ee-8843-24622e1c70bb admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:35.064568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b729f-f643-42db-bad1-7ffe15d41561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:35.065278 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b729f-f643-42db-bad1-7ffe15d41561 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:35.065659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b729f-f643-42db-bad1-7ffe15d41561 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:35.066148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b729f-f643-42db-bad1-7ffe15d41561 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:35.066987 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b729f-f643-42db-bad1-7ffe15d41561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:35.069886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b729f-f643-42db-bad1-7ffe15d41561 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:35.072016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1277/2532] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:35 2022] PUT /identity/v3/projects/a05914c4ddfa4c72b485a28548b9d83e/users/6c2457e56a1841c0907777cbf6284921/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:57:35.114676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1256/2533] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:35 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:57:35.117125 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200441d5-86f2-46ee-8843-24622e1c70bb admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:35.143938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:35.197528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200441d5-86f2-46ee-8843-24622e1c70bb admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:35.269360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0cbe9-7f8a-462f-a38e-4b93f33acf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:35.271023 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0cbe9-7f8a-462f-a38e-4b93f33acf8e admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:35.271339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0cbe9-7f8a-462f-a38e-4b93f33acf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:35.271619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0cbe9-7f8a-462f-a38e-4b93f33acf8e admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:35.272387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0cbe9-7f8a-462f-a38e-4b93f33acf8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:35.274570 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0cbe9-7f8a-462f-a38e-4b93f33acf8e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:35.303817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1278/2534] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:35 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:35.308223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b729f-f643-42db-bad1-7ffe15d41561 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:35.319380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b729f-f643-42db-bad1-7ffe15d41561 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:35.325233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6aa693-3f99-4146-a1de-8092fe1f6ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:35.325876 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aa693-3f99-4146-a1de-8092fe1f6ded admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:35.326163 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aa693-3f99-4146-a1de-8092fe1f6ded admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:35.326432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6aa693-3f99-4146-a1de-8092fe1f6ded admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:35.327159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6aa693-3f99-4146-a1de-8092fe1f6ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:35.328768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6aa693-3f99-4146-a1de-8092fe1f6ded admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:35.339696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1257/2535] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:35.343794 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0cbe9-7f8a-462f-a38e-4b93f33acf8e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:35.356350 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0cbe9-7f8a-462f-a38e-4b93f33acf8e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:35.361555 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53efce6-5b1a-4f29-9b0f-1578c28d5d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:35.362246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53efce6-5b1a-4f29-9b0f-1578c28d5d97 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:35.362597 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53efce6-5b1a-4f29-9b0f-1578c28d5d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:35.362961 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53efce6-5b1a-4f29-9b0f-1578c28d5d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:35.363663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53efce6-5b1a-4f29-9b0f-1578c28d5d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:35.365323 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53efce6-5b1a-4f29-9b0f-1578c28d5d97 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:35.376092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1279/2536] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:35.380521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6aa693-3f99-4146-a1de-8092fe1f6ded admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:35.390670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6aa693-3f99-4146-a1de-8092fe1f6ded admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:35.395255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b995248-9f35-40c9-8feb-b4041bbb3d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:35.396593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b995248-9f35-40c9-8feb-b4041bbb3d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:35.396980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b995248-9f35-40c9-8feb-b4041bbb3d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:35.397258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b995248-9f35-40c9-8feb-b4041bbb3d1e admin admin] PATH_INFO: `/v3/projects/eb51bc2bb5ba4a018af6f83ea1d95112/users/8a68ac1cfb2e4b80a69d489780323f01/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:35.412488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b995248-9f35-40c9-8feb-b4041bbb3d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb51bc2bb5ba4a018af6f83ea1d95112, user_id=8a68ac1cfb2e4b80a69d489780323f01, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:35.416013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b995248-9f35-40c9-8feb-b4041bbb3d1e admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:35.434732 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1258/2537] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:35 2022] PUT /identity/v3/projects/eb51bc2bb5ba4a018af6f83ea1d95112/users/8a68ac1cfb2e4b80a69d489780323f01/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:35.444990 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de01ca-c246-4137-b0b3-a289241cddba None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:35.445397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de01ca-c246-4137-b0b3-a289241cddba None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:35.445844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de01ca-c246-4137-b0b3-a289241cddba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:35.508981 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-45de01ca-c246-4137-b0b3-a289241cddba None None] MFA Rules not processed for user `8a68ac1cfb2e4b80a69d489780323f01`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:35.510212 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-45de01ca-c246-4137-b0b3-a289241cddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:35.589969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1280/2538] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:35.602655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b995248-9f35-40c9-8feb-b4041bbb3d1e admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:35.604992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6b995248-9f35-40c9-8feb-b4041bbb3d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:35.679944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:35.698898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b995248-9f35-40c9-8feb-b4041bbb3d1e admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:35.707059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91052090-ee4e-4deb-9413-5802c6de0546 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:35.708050 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91052090-ee4e-4deb-9413-5802c6de0546 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:35.708470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91052090-ee4e-4deb-9413-5802c6de0546 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:35.708911 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91052090-ee4e-4deb-9413-5802c6de0546 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:35.719880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91052090-ee4e-4deb-9413-5802c6de0546 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:35.721488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91052090-ee4e-4deb-9413-5802c6de0546 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:35.742695 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1259/2539] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:38.446124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a631d-6697-4655-92d4-b408efd418b5 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:38.447145 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a631d-6697-4655-92d4-b408efd418b5 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:38.447634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a631d-6697-4655-92d4-b408efd418b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:38.508838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-844a631d-6697-4655-92d4-b408efd418b5 None None] MFA Rules not processed for user `19c2f15eb1644278b911cc50370da9b7`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:38.516865 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-844a631d-6697-4655-92d4-b408efd418b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:38.544159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:38.638673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1281/2540] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:40.424056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad68010-9168-4ede-83af-6c7bf37878ba None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:40.424555 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad68010-9168-4ede-83af-6c7bf37878ba None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:40.424707 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad68010-9168-4ede-83af-6c7bf37878ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:40.454682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-8ad68010-9168-4ede-83af-6c7bf37878ba None None] MFA Rules not processed for user `6c2457e56a1841c0907777cbf6284921`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:40.455896 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-8ad68010-9168-4ede-83af-6c7bf37878ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:40.546567 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1260/2541] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:40.563223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a631d-6697-4655-92d4-b408efd418b5 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:40.565064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-844a631d-6697-4655-92d4-b408efd418b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:40.640756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a631d-6697-4655-92d4-b408efd418b5 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:40.646680 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596216cd-96b5-470a-a787-b0a82c49bf97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:40.647312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596216cd-96b5-470a-a787-b0a82c49bf97 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:40.647593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596216cd-96b5-470a-a787-b0a82c49bf97 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:40.647863 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596216cd-96b5-470a-a787-b0a82c49bf97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:40.649693 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-596216cd-96b5-470a-a787-b0a82c49bf97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:40.738658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596216cd-96b5-470a-a787-b0a82c49bf97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:40.741059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596216cd-96b5-470a-a787-b0a82c49bf97 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:40.753740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1282/2542] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:42.471488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771d655-b727-4f38-8937-08dc922f7040 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:42.472190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771d655-b727-4f38-8937-08dc922f7040 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:42.472190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771d655-b727-4f38-8937-08dc922f7040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:42.541367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-a771d655-b727-4f38-8937-08dc922f7040 None None] MFA Rules not processed for user `802ed5c51d184f6c925f31ab6020ac93`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:42.545300 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a771d655-b727-4f38-8937-08dc922f7040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:42.572837 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1261/2543] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:57:43.753891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596216cd-96b5-470a-a787-b0a82c49bf97 service nova] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:43.821750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:43.839746 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596216cd-96b5-470a-a787-b0a82c49bf97 service nova] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:43.844346 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:43.844986 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:43.845269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:43.845543 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] PATH_INFO: `/v3/users/8a68ac1cfb2e4b80a69d489780323f01` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:43.846746 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a68ac1cfb2e4b80a69d489780323f01)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:43.849371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:43.905020 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a68ac1cfb2e4b80a69d489780323f01'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:43.918081 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a68ac1cfb2e4b80a69d489780323f01'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:43.926758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a68ac1cfb2e4b80a69d489780323f01'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:43.928974 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a68ac1cfb2e4b80a69d489780323f01'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:43.929707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a68ac1cfb2e4b80a69d489780323f01'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:43.943522 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1283/2544] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:57:43 2022] DELETE /identity/v3/users/8a68ac1cfb2e4b80a69d489780323f01 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:43.948849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af77ec-ee1c-4c1f-aa1d-06b8e94e4cfc None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:43.949482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af77ec-ee1c-4c1f-aa1d-06b8e94e4cfc None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:43.949482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af77ec-ee1c-4c1f-aa1d-06b8e94e4cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:43.994512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-e8af77ec-ee1c-4c1f-aa1d-06b8e94e4cfc None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:43.995793 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e8af77ec-ee1c-4c1f-aa1d-06b8e94e4cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:44.061250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1262/2545] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:44.071880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:44.073284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:44.136454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a0407-ef2b-477f-b332-63dc763522ff admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:44.140148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb308cc0-ef34-4c38-acc2-b67e0fe6dce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:44.140758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb308cc0-ef34-4c38-acc2-b67e0fe6dce0 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:44.141032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb308cc0-ef34-4c38-acc2-b67e0fe6dce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:44.141301 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb308cc0-ef34-4c38-acc2-b67e0fe6dce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:44.151640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb308cc0-ef34-4c38-acc2-b67e0fe6dce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:44.153681 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb308cc0-ef34-4c38-acc2-b67e0fe6dce0 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:44.167394 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1284/2546] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:44.502467 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af77ec-ee1c-4c1f-aa1d-06b8e94e4cfc None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:44.503969 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e8af77ec-ee1c-4c1f-aa1d-06b8e94e4cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:44.518593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af77ec-ee1c-4c1f-aa1d-06b8e94e4cfc None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:44.524367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83847298-bebf-45ee-86c7-293f8339bb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:44.524995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83847298-bebf-45ee-86c7-293f8339bb5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:44.525274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83847298-bebf-45ee-86c7-293f8339bb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:44.525547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83847298-bebf-45ee-86c7-293f8339bb5f admin admin] PATH_INFO: `/v3/projects/eb51bc2bb5ba4a018af6f83ea1d95112` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:44.527041 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83847298-bebf-45ee-86c7-293f8339bb5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb51bc2bb5ba4a018af6f83ea1d95112)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:44.529288 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83847298-bebf-45ee-86c7-293f8339bb5f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:44.578866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:44.595750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-83847298-bebf-45ee-86c7-293f8339bb5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb51bc2bb5ba4a018af6f83ea1d95112', 'request_id': 'req-83847298-bebf-45ee-86c7-293f8339bb5f'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:44.603547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-83847298-bebf-45ee-86c7-293f8339bb5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb51bc2bb5ba4a018af6f83ea1d95112', 'request_id': 'req-83847298-bebf-45ee-86c7-293f8339bb5f'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:44.612850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-83847298-bebf-45ee-86c7-293f8339bb5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb51bc2bb5ba4a018af6f83ea1d95112', 'request_id': 'req-83847298-bebf-45ee-86c7-293f8339bb5f'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:44.625829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-83847298-bebf-45ee-86c7-293f8339bb5f admin admin] The token cache is being invalidate because project eb51bc2bb5ba4a018af6f83ea1d95112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:57:44.626118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-83847298-bebf-45ee-86c7-293f8339bb5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:44.626812 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-83847298-bebf-45ee-86c7-293f8339bb5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:44.628338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1263/2547] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:57:44 2022] DELETE /identity/v3/projects/eb51bc2bb5ba4a018af6f83ea1d95112 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:44.633359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d548cd-0eff-42ad-a1b9-a2bd258b2d23 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:44.633791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d548cd-0eff-42ad-a1b9-a2bd258b2d23 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:44.634103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d548cd-0eff-42ad-a1b9-a2bd258b2d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:44.697435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e1d548cd-0eff-42ad-a1b9-a2bd258b2d23 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:44.698745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e1d548cd-0eff-42ad-a1b9-a2bd258b2d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:44.754868 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:44.782679 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1285/2548] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:44.788804 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb823aba-c570-4e4d-b0d8-df7598bc4a53 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:44.789355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb823aba-c570-4e4d-b0d8-df7598bc4a53 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:44.789832 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb823aba-c570-4e4d-b0d8-df7598bc4a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:44.814891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-fb823aba-c570-4e4d-b0d8-df7598bc4a53 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:44.816203 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-fb823aba-c570-4e4d-b0d8-df7598bc4a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:44.826233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1264/2549] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:44.830473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d548cd-0eff-42ad-a1b9-a2bd258b2d23 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:44.844356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d548cd-0eff-42ad-a1b9-a2bd258b2d23 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:44.851137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece00f0-f6f2-472c-8aa3-8fbab3dd8db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:44.855665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece00f0-f6f2-472c-8aa3-8fbab3dd8db0 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:44.856049 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece00f0-f6f2-472c-8aa3-8fbab3dd8db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:44.857211 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece00f0-f6f2-472c-8aa3-8fbab3dd8db0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:44.858053 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece00f0-f6f2-472c-8aa3-8fbab3dd8db0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:57:44.858428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece00f0-f6f2-472c-8aa3-8fbab3dd8db0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:44.860434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece00f0-f6f2-472c-8aa3-8fbab3dd8db0 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:44.883844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1286/2550] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:57:44.885338 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb823aba-c570-4e4d-b0d8-df7598bc4a53 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:44.896336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb823aba-c570-4e4d-b0d8-df7598bc4a53 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:44.906116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f5d06-6efd-4b89-b939-653a8ae32823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:44.906884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f5d06-6efd-4b89-b939-653a8ae32823 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:44.907224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f5d06-6efd-4b89-b939-653a8ae32823 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:44.907554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f5d06-6efd-4b89-b939-653a8ae32823 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:44.908419 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f5d06-6efd-4b89-b939-653a8ae32823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:44.911419 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f5d06-6efd-4b89-b939-653a8ae32823 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:44.941253 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1265/2551] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:44 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:57:44.945831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece00f0-f6f2-472c-8aa3-8fbab3dd8db0 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:44.956991 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece00f0-f6f2-472c-8aa3-8fbab3dd8db0 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:45.024194 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1bd2e-45aa-4178-9281-f5cd54263341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:45.024935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1bd2e-45aa-4178-9281-f5cd54263341 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:45.025280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1bd2e-45aa-4178-9281-f5cd54263341 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:45.025617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1bd2e-45aa-4178-9281-f5cd54263341 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:45.026356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1bd2e-45aa-4178-9281-f5cd54263341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:45.028531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1bd2e-45aa-4178-9281-f5cd54263341 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:45.073715 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1287/2552] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:44 2022] POST /identity/v3/users => generated 285 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:45.078511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f5d06-6efd-4b89-b939-653a8ae32823 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:45.090431 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f5d06-6efd-4b89-b939-653a8ae32823 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:45.099648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f3284-ca28-4dfc-a06a-095e8d547463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:45.100438 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f3284-ca28-4dfc-a06a-095e8d547463 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:45.100846 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f3284-ca28-4dfc-a06a-095e8d547463 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:45.101303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f3284-ca28-4dfc-a06a-095e8d547463 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:45.102110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f3284-ca28-4dfc-a06a-095e8d547463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:45.104129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f3284-ca28-4dfc-a06a-095e8d547463 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:45.116249 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1266/2553] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:45.121256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1bd2e-45aa-4178-9281-f5cd54263341 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:45.137632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1bd2e-45aa-4178-9281-f5cd54263341 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:45.145665 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427210e-9dce-4a73-9b93-7af02d020c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:45.147364 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427210e-9dce-4a73-9b93-7af02d020c81 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:45.147533 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427210e-9dce-4a73-9b93-7af02d020c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:45.147702 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427210e-9dce-4a73-9b93-7af02d020c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:45.148405 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427210e-9dce-4a73-9b93-7af02d020c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:45.150024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427210e-9dce-4a73-9b93-7af02d020c81 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:45.161759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1288/2554] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:45.165828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f3284-ca28-4dfc-a06a-095e8d547463 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:45.180477 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f3284-ca28-4dfc-a06a-095e8d547463 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:45.187021 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba7d8c-7ba0-48ca-abb6-770e918d240a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:45.187712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba7d8c-7ba0-48ca-abb6-770e918d240a admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:45.187986 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba7d8c-7ba0-48ca-abb6-770e918d240a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:45.188437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba7d8c-7ba0-48ca-abb6-770e918d240a admin admin] PATH_INFO: `/v3/projects/da445bb412bb456799a4410ccaf7e2ec/users/5615673b7a3843c5aa200bdf501d4d27/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:45.205520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba7d8c-7ba0-48ca-abb6-770e918d240a admin admin] RBAC: Authorizing `identity:create_grant(project_id=da445bb412bb456799a4410ccaf7e2ec, user_id=5615673b7a3843c5aa200bdf501d4d27, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:45.209435 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba7d8c-7ba0-48ca-abb6-770e918d240a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:45.231989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1267/2555] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:45 2022] PUT /identity/v3/projects/da445bb412bb456799a4410ccaf7e2ec/users/5615673b7a3843c5aa200bdf501d4d27/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:45.246377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e4f5d-19ce-4751-ba7e-5d64940d520c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:45.246868 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e4f5d-19ce-4751-ba7e-5d64940d520c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:45.247351 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e4f5d-19ce-4751-ba7e-5d64940d520c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:45.292194 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-825e4f5d-19ce-4751-ba7e-5d64940d520c None None] MFA Rules not processed for user `5615673b7a3843c5aa200bdf501d4d27`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:45.294816 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-825e4f5d-19ce-4751-ba7e-5d64940d520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:45.427451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1289/2556] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:45.430719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba7d8c-7ba0-48ca-abb6-770e918d240a admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:45.503289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba7d8c-7ba0-48ca-abb6-770e918d240a admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:45.508889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f515d36-2c27-4226-b914-90d476d149c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:45.509982 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f515d36-2c27-4226-b914-90d476d149c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:45.511005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f515d36-2c27-4226-b914-90d476d149c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:45.511511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f515d36-2c27-4226-b914-90d476d149c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:45.512419 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f515d36-2c27-4226-b914-90d476d149c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:45.516578 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f515d36-2c27-4226-b914-90d476d149c6 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:45.556255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1268/2557] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:45 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:45.560467 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e4f5d-19ce-4751-ba7e-5d64940d520c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:45.575588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e4f5d-19ce-4751-ba7e-5d64940d520c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:45.642306 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:45.670868 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16583dd-e44f-4d73-8b76-fabb7d6c49b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:45.671834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16583dd-e44f-4d73-8b76-fabb7d6c49b3 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:45.672208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16583dd-e44f-4d73-8b76-fabb7d6c49b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:45.672544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16583dd-e44f-4d73-8b76-fabb7d6c49b3 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:45.673279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16583dd-e44f-4d73-8b76-fabb7d6c49b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:45.675751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16583dd-e44f-4d73-8b76-fabb7d6c49b3 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:45.709056 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1290/2558] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:45 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:57:45.713101 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f515d36-2c27-4226-b914-90d476d149c6 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:45.723927 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f515d36-2c27-4226-b914-90d476d149c6 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:45.729853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829e6f3-b25b-41b6-9236-f8d9c24edd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:45.730488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829e6f3-b25b-41b6-9236-f8d9c24edd78 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:45.730809 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829e6f3-b25b-41b6-9236-f8d9c24edd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:45.732058 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829e6f3-b25b-41b6-9236-f8d9c24edd78 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:45.732767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829e6f3-b25b-41b6-9236-f8d9c24edd78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:45.734397 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829e6f3-b25b-41b6-9236-f8d9c24edd78 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:45.749483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1269/2559] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:45.754269 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16583dd-e44f-4d73-8b76-fabb7d6c49b3 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:45.769242 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16583dd-e44f-4d73-8b76-fabb7d6c49b3 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:45.774691 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f9435-472d-4865-a075-20f242eee066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:45.775476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f9435-472d-4865-a075-20f242eee066 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:45.775751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f9435-472d-4865-a075-20f242eee066 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:45.776368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f9435-472d-4865-a075-20f242eee066 admin admin] PATH_INFO: `/v3/projects/a90bd587b7434b88919007f0f058c038/users/bad1292679d545a089af3ee7d110c159/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:45.799133 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f9435-472d-4865-a075-20f242eee066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a90bd587b7434b88919007f0f058c038, user_id=bad1292679d545a089af3ee7d110c159, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:45.803349 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f9435-472d-4865-a075-20f242eee066 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:45.835560 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1291/2560] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:45 2022] PUT /identity/v3/projects/a90bd587b7434b88919007f0f058c038/users/bad1292679d545a089af3ee7d110c159/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:45.840503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829e6f3-b25b-41b6-9236-f8d9c24edd78 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:45.858024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829e6f3-b25b-41b6-9236-f8d9c24edd78 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:45.952514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:45.996319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cfcab-6a52-4ca0-ba65-01f87ab9efe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:45.996986 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cfcab-6a52-4ca0-ba65-01f87ab9efe2 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:45.997263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cfcab-6a52-4ca0-ba65-01f87ab9efe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:45.997560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cfcab-6a52-4ca0-ba65-01f87ab9efe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:45.998262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cfcab-6a52-4ca0-ba65-01f87ab9efe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:46.000173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cfcab-6a52-4ca0-ba65-01f87ab9efe2 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:46.021852 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1270/2561] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:46.026707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f9435-472d-4865-a075-20f242eee066 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:46.051605 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f9435-472d-4865-a075-20f242eee066 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:46.059488 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46addda3-07be-406c-ae65-b1f9c7c70d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:46.060406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46addda3-07be-406c-ae65-b1f9c7c70d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:46.060807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46addda3-07be-406c-ae65-b1f9c7c70d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:46.061198 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46addda3-07be-406c-ae65-b1f9c7c70d2a admin admin] PATH_INFO: `/v3/projects/a90bd587b7434b88919007f0f058c038/users/bad1292679d545a089af3ee7d110c159/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:46.063454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46addda3-07be-406c-ae65-b1f9c7c70d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a90bd587b7434b88919007f0f058c038, user_id=bad1292679d545a089af3ee7d110c159, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:46.066418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46addda3-07be-406c-ae65-b1f9c7c70d2a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:46.090910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1292/2562] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:46 2022] PUT /identity/v3/projects/a90bd587b7434b88919007f0f058c038/users/bad1292679d545a089af3ee7d110c159/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:46.093312 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ffad-1520-4085-887d-b0348adad154 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:46.093782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ffad-1520-4085-887d-b0348adad154 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:46.093782 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3ffad-1520-4085-887d-b0348adad154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:46.174417 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-7ad3ffad-1520-4085-887d-b0348adad154 None None] MFA Rules not processed for user `bad1292679d545a089af3ee7d110c159`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:46.175983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7ad3ffad-1520-4085-887d-b0348adad154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:46.336849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1271/2563] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:46.354968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46addda3-07be-406c-ae65-b1f9c7c70d2a admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:46.356718 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-46addda3-07be-406c-ae65-b1f9c7c70d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:46.487256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46addda3-07be-406c-ae65-b1f9c7c70d2a admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:46.491562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc7a86-09b7-45c2-b4cb-5d02259f71a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:46.492596 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc7a86-09b7-45c2-b4cb-5d02259f71a0 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:46.492972 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc7a86-09b7-45c2-b4cb-5d02259f71a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:46.493315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc7a86-09b7-45c2-b4cb-5d02259f71a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:46.508164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc7a86-09b7-45c2-b4cb-5d02259f71a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:46.510721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc7a86-09b7-45c2-b4cb-5d02259f71a0 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:46.525058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1293/2564] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:46.966963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3ffad-1520-4085-887d-b0348adad154 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:46.969133 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7ad3ffad-1520-4085-887d-b0348adad154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:47.113274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3ffad-1520-4085-887d-b0348adad154 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:47.122717 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb555103-34d5-4505-9cb3-5daf96b37dfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:47.123422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb555103-34d5-4505-9cb3-5daf96b37dfd service placement] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:47.123613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb555103-34d5-4505-9cb3-5daf96b37dfd service placement] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:47.123885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb555103-34d5-4505-9cb3-5daf96b37dfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:47.125666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-bb555103-34d5-4505-9cb3-5daf96b37dfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:47.142065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb555103-34d5-4505-9cb3-5daf96b37dfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:47.143920 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb555103-34d5-4505-9cb3-5daf96b37dfd service placement] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:47.157340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1272/2565] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:48.428889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc7a86-09b7-45c2-b4cb-5d02259f71a0 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:48.509897 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc7a86-09b7-45c2-b4cb-5d02259f71a0 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:48.514683 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:48.515401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:48.515678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:48.515951 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] PATH_INFO: `/v3/users/5615673b7a3843c5aa200bdf501d4d27` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:48.517127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5615673b7a3843c5aa200bdf501d4d27)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:48.519106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:48.570482 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:48.576232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5615673b7a3843c5aa200bdf501d4d27'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:48.591791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5615673b7a3843c5aa200bdf501d4d27'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:48.607298 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5615673b7a3843c5aa200bdf501d4d27'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:48.608206 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5615673b7a3843c5aa200bdf501d4d27'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:48.608913 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5615673b7a3843c5aa200bdf501d4d27'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:48.631409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1294/2566] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:57:48 2022] DELETE /identity/v3/users/5615673b7a3843c5aa200bdf501d4d27 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:48.636188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d794d-31d3-4fbd-aed1-3f2d8a39a541 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:48.636738 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d794d-31d3-4fbd-aed1-3f2d8a39a541 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:48.636832 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d794d-31d3-4fbd-aed1-3f2d8a39a541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:48.659224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-202d794d-31d3-4fbd-aed1-3f2d8a39a541 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:48.660573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-202d794d-31d3-4fbd-aed1-3f2d8a39a541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:48.684285 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:48.726869 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1273/2567] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:48.736799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:48.738228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:48.802424 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304935b7-c421-404e-8cad-29d18e94bc46 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:48.809977 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ea5e0-1780-4ca7-9e68-6de0eaf96b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:48.810538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ea5e0-1780-4ca7-9e68-6de0eaf96b0e service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:48.810841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ea5e0-1780-4ca7-9e68-6de0eaf96b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:48.811113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ea5e0-1780-4ca7-9e68-6de0eaf96b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:48.821575 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ea5e0-1780-4ca7-9e68-6de0eaf96b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:48.823049 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ea5e0-1780-4ca7-9e68-6de0eaf96b0e service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:48.832882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1295/2568] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:49.271454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d794d-31d3-4fbd-aed1-3f2d8a39a541 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:49.273187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-202d794d-31d3-4fbd-aed1-3f2d8a39a541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:49.288055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d794d-31d3-4fbd-aed1-3f2d8a39a541 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:49.312324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:49.312969 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:49.313203 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:49.313683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] PATH_INFO: `/v3/projects/da445bb412bb456799a4410ccaf7e2ec` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:49.316263 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] RBAC: Authorizing `identity:delete_project(project_id=da445bb412bb456799a4410ccaf7e2ec)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:49.319689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:49.399643 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da445bb412bb456799a4410ccaf7e2ec', 'request_id': 'req-efdd466d-47c3-4586-8059-644b64539adc'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:49.405970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da445bb412bb456799a4410ccaf7e2ec', 'request_id': 'req-efdd466d-47c3-4586-8059-644b64539adc'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:49.415679 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da445bb412bb456799a4410ccaf7e2ec', 'request_id': 'req-efdd466d-47c3-4586-8059-644b64539adc'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:49.427639 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] The token cache is being invalidate because project da445bb412bb456799a4410ccaf7e2ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:57:49.428832 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:49.429502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:49.431072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1274/2569] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:57:49 2022] DELETE /identity/v3/projects/da445bb412bb456799a4410ccaf7e2ec => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:49.435081 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ea5e0-1780-4ca7-9e68-6de0eaf96b0e service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:49.445801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ea5e0-1780-4ca7-9e68-6de0eaf96b0e service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:49.517908 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d97533-bbd2-4203-8b9f-aff7ee23806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:49.518548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d97533-bbd2-4203-8b9f-aff7ee23806e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:49.519196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d97533-bbd2-4203-8b9f-aff7ee23806e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:49.519196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d97533-bbd2-4203-8b9f-aff7ee23806e admin admin] PATH_INFO: `/v3/users/bad1292679d545a089af3ee7d110c159` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:49.520404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d97533-bbd2-4203-8b9f-aff7ee23806e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad1292679d545a089af3ee7d110c159)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:49.523363 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d97533-bbd2-4203-8b9f-aff7ee23806e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:49.585932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-17d97533-bbd2-4203-8b9f-aff7ee23806e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad1292679d545a089af3ee7d110c159'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:49.597415 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:49.600737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-17d97533-bbd2-4203-8b9f-aff7ee23806e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad1292679d545a089af3ee7d110c159'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:49.609355 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-17d97533-bbd2-4203-8b9f-aff7ee23806e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad1292679d545a089af3ee7d110c159'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:49.610284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-17d97533-bbd2-4203-8b9f-aff7ee23806e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad1292679d545a089af3ee7d110c159'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:49.611278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-17d97533-bbd2-4203-8b9f-aff7ee23806e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad1292679d545a089af3ee7d110c159'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:49.627937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1296/2570] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:57:49 2022] DELETE /identity/v3/users/bad1292679d545a089af3ee7d110c159 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:57:49.969739 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:49.971797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:49.986669 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd466d-47c3-4586-8059-644b64539adc admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:50.062486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6e827-c2bd-4964-8153-5be78dc1c12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:50.063179 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6e827-c2bd-4964-8153-5be78dc1c12f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:50.063458 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6e827-c2bd-4964-8153-5be78dc1c12f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:50.063739 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6e827-c2bd-4964-8153-5be78dc1c12f admin admin] PATH_INFO: `/v3/projects/a90bd587b7434b88919007f0f058c038` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:50.064962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6e827-c2bd-4964-8153-5be78dc1c12f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a90bd587b7434b88919007f0f058c038)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:50.066976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6e827-c2bd-4964-8153-5be78dc1c12f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:50.123522 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-93d6e827-c2bd-4964-8153-5be78dc1c12f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a90bd587b7434b88919007f0f058c038', 'request_id': 'req-93d6e827-c2bd-4964-8153-5be78dc1c12f'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:50.127719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:50.131659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-93d6e827-c2bd-4964-8153-5be78dc1c12f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a90bd587b7434b88919007f0f058c038', 'request_id': 'req-93d6e827-c2bd-4964-8153-5be78dc1c12f'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:50.141683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-93d6e827-c2bd-4964-8153-5be78dc1c12f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a90bd587b7434b88919007f0f058c038', 'request_id': 'req-93d6e827-c2bd-4964-8153-5be78dc1c12f'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:50.154173 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-93d6e827-c2bd-4964-8153-5be78dc1c12f admin admin] The token cache is being invalidate because project a90bd587b7434b88919007f0f058c038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:57:50.154448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-93d6e827-c2bd-4964-8153-5be78dc1c12f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:50.155148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-93d6e827-c2bd-4964-8153-5be78dc1c12f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:50.156664 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1275/2571] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:57:49 2022] DELETE /identity/v3/projects/a90bd587b7434b88919007f0f058c038 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:50.162613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fb711-905e-40e4-b4fd-400fa844ade7 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:50.162972 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fb711-905e-40e4-b4fd-400fa844ade7 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:50.163304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fb711-905e-40e4-b4fd-400fa844ade7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:50.185999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-f85fb711-905e-40e4-b4fd-400fa844ade7 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:50.187261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f85fb711-905e-40e4-b4fd-400fa844ade7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:50.253104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1297/2572] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:50.261436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494a282-c688-4302-9556-017321e71bfd None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:50.261731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494a282-c688-4302-9556-017321e71bfd None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:50.262007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494a282-c688-4302-9556-017321e71bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:50.285997 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-f494a282-c688-4302-9556-017321e71bfd None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:50.288575 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f494a282-c688-4302-9556-017321e71bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:50.301421 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1276/2573] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:50.308221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fb711-905e-40e4-b4fd-400fa844ade7 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:50.320135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fb711-905e-40e4-b4fd-400fa844ade7 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:50.326100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c6934-fcd3-4e62-8cd1-6a4c05f53746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:50.327923 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c6934-fcd3-4e62-8cd1-6a4c05f53746 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:50.328238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c6934-fcd3-4e62-8cd1-6a4c05f53746 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:50.328491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c6934-fcd3-4e62-8cd1-6a4c05f53746 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:50.330172 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c6934-fcd3-4e62-8cd1-6a4c05f53746 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:57:50.330463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c6934-fcd3-4e62-8cd1-6a4c05f53746 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:50.337241 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c6934-fcd3-4e62-8cd1-6a4c05f53746 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:50.350278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1298/2574] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:57:50.354745 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494a282-c688-4302-9556-017321e71bfd None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:50.371858 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494a282-c688-4302-9556-017321e71bfd None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:50.376767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599aa4f0-ee2e-4404-a1cc-cf0bacb1827a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:50.377388 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599aa4f0-ee2e-4404-a1cc-cf0bacb1827a admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:50.377665 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599aa4f0-ee2e-4404-a1cc-cf0bacb1827a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:50.377938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599aa4f0-ee2e-4404-a1cc-cf0bacb1827a admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:50.378615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599aa4f0-ee2e-4404-a1cc-cf0bacb1827a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:50.381055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599aa4f0-ee2e-4404-a1cc-cf0bacb1827a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:50.424440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1277/2575] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:50 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:50.430285 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c6934-fcd3-4e62-8cd1-6a4c05f53746 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:50.444600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c6934-fcd3-4e62-8cd1-6a4c05f53746 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:50.518844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa90a79-49d5-4bc7-8383-deb1d20f1ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:50.519467 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa90a79-49d5-4bc7-8383-deb1d20f1ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:50.519747 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa90a79-49d5-4bc7-8383-deb1d20f1ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:50.520019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa90a79-49d5-4bc7-8383-deb1d20f1ac3 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:50.520684 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa90a79-49d5-4bc7-8383-deb1d20f1ac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:50.522791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa90a79-49d5-4bc7-8383-deb1d20f1ac3 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:50.549711 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1299/2576] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:50 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:50.555940 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599aa4f0-ee2e-4404-a1cc-cf0bacb1827a admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:50.572202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599aa4f0-ee2e-4404-a1cc-cf0bacb1827a admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:50.579653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf439654-02ea-4cf5-831f-61bd376f3bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:50.581040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf439654-02ea-4cf5-831f-61bd376f3bee admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:50.581040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf439654-02ea-4cf5-831f-61bd376f3bee admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:50.581040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf439654-02ea-4cf5-831f-61bd376f3bee admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:50.581897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf439654-02ea-4cf5-831f-61bd376f3bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:50.584446 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf439654-02ea-4cf5-831f-61bd376f3bee admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:50.596475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1278/2577] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:50.600428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa90a79-49d5-4bc7-8383-deb1d20f1ac3 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:50.614061 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa90a79-49d5-4bc7-8383-deb1d20f1ac3 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:50.619798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40e96d-32ac-4e3d-a2b5-9dad01c7b3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:50.620480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40e96d-32ac-4e3d-a2b5-9dad01c7b3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:50.620757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40e96d-32ac-4e3d-a2b5-9dad01c7b3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:50.621026 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40e96d-32ac-4e3d-a2b5-9dad01c7b3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:50.621724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40e96d-32ac-4e3d-a2b5-9dad01c7b3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:50.623385 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40e96d-32ac-4e3d-a2b5-9dad01c7b3e7 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:50.638796 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1300/2578] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:50.640297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf439654-02ea-4cf5-831f-61bd376f3bee admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:50.654052 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf439654-02ea-4cf5-831f-61bd376f3bee admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:50.659109 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bec962-8a01-497d-99b6-d90e07d28d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:50.659770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bec962-8a01-497d-99b6-d90e07d28d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:50.660046 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bec962-8a01-497d-99b6-d90e07d28d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:50.660315 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bec962-8a01-497d-99b6-d90e07d28d70 admin admin] PATH_INFO: `/v3/projects/2ec668f9c77a44d59149ed10900dfec1/users/c21891153f6141e09f39ae05a3707eb7/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:50.675988 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bec962-8a01-497d-99b6-d90e07d28d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ec668f9c77a44d59149ed10900dfec1, user_id=c21891153f6141e09f39ae05a3707eb7, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:50.680087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bec962-8a01-497d-99b6-d90e07d28d70 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:50.699760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1279/2579] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:50 2022] PUT /identity/v3/projects/2ec668f9c77a44d59149ed10900dfec1/users/c21891153f6141e09f39ae05a3707eb7/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:57:50.714979 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c2496-ac53-4445-8f16-7d72c650732d None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:50.715266 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c2496-ac53-4445-8f16-7d72c650732d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:50.715538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c2496-ac53-4445-8f16-7d72c650732d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:50.751112 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-fa3c2496-ac53-4445-8f16-7d72c650732d None None] MFA Rules not processed for user `c21891153f6141e09f39ae05a3707eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:50.752331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fa3c2496-ac53-4445-8f16-7d72c650732d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:50.788949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:50.831092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1301/2580] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:50.837007 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bec962-8a01-497d-99b6-d90e07d28d70 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:50.918307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bec962-8a01-497d-99b6-d90e07d28d70 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:50.923244 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d716537-60d7-43c5-a8c4-e2dd4b17c3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:50.923891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d716537-60d7-43c5-a8c4-e2dd4b17c3cd admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:50.924223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d716537-60d7-43c5-a8c4-e2dd4b17c3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:50.924525 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d716537-60d7-43c5-a8c4-e2dd4b17c3cd admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:50.925161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d716537-60d7-43c5-a8c4-e2dd4b17c3cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:50.927830 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d716537-60d7-43c5-a8c4-e2dd4b17c3cd admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:50.962670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1280/2581] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:50 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:57:50.965157 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c2496-ac53-4445-8f16-7d72c650732d None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:50.976806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c2496-ac53-4445-8f16-7d72c650732d None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:51.038291 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884ef55-90ff-436a-afd5-ccb368335eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:51.039221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884ef55-90ff-436a-afd5-ccb368335eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:51.039491 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884ef55-90ff-436a-afd5-ccb368335eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:51.039768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884ef55-90ff-436a-afd5-ccb368335eb2 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:51.040485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884ef55-90ff-436a-afd5-ccb368335eb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:51.042807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884ef55-90ff-436a-afd5-ccb368335eb2 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:51.065122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1302/2582] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:50 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:57:51.069795 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d716537-60d7-43c5-a8c4-e2dd4b17c3cd admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:51.080542 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d716537-60d7-43c5-a8c4-e2dd4b17c3cd admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:51.086959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791d16f-114c-4731-9964-8515240535c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:51.087596 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791d16f-114c-4731-9964-8515240535c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:51.087875 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791d16f-114c-4731-9964-8515240535c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:51.088145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791d16f-114c-4731-9964-8515240535c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:51.088850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791d16f-114c-4731-9964-8515240535c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:51.090457 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791d16f-114c-4731-9964-8515240535c7 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:51.102148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1281/2583] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:51.106316 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884ef55-90ff-436a-afd5-ccb368335eb2 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:51.116510 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884ef55-90ff-436a-afd5-ccb368335eb2 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:51.121464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e17cbc-c302-4caf-b3f9-4d8c2ad1d2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:51.122063 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e17cbc-c302-4caf-b3f9-4d8c2ad1d2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:51.122342 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e17cbc-c302-4caf-b3f9-4d8c2ad1d2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:51.122612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e17cbc-c302-4caf-b3f9-4d8c2ad1d2f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:51.123334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e17cbc-c302-4caf-b3f9-4d8c2ad1d2f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:51.125018 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e17cbc-c302-4caf-b3f9-4d8c2ad1d2f9 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:51.135635 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1303/2584] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:51.139791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791d16f-114c-4731-9964-8515240535c7 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:51.150365 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791d16f-114c-4731-9964-8515240535c7 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:51.155578 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627e1af-2e98-4d3f-8acc-b8ff56c3f5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:51.156234 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627e1af-2e98-4d3f-8acc-b8ff56c3f5f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:51.156512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627e1af-2e98-4d3f-8acc-b8ff56c3f5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:51.156784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627e1af-2e98-4d3f-8acc-b8ff56c3f5f6 admin admin] PATH_INFO: `/v3/projects/f52736554ec841fc8d65ee3c04bb55ac/users/583b7ec1b1af4d8d8709fcc017b2d153/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:51.173107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627e1af-2e98-4d3f-8acc-b8ff56c3f5f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f52736554ec841fc8d65ee3c04bb55ac, user_id=583b7ec1b1af4d8d8709fcc017b2d153, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:51.175834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627e1af-2e98-4d3f-8acc-b8ff56c3f5f6 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:51.192336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1282/2585] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:51 2022] PUT /identity/v3/projects/f52736554ec841fc8d65ee3c04bb55ac/users/583b7ec1b1af4d8d8709fcc017b2d153/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:51.204918 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b7392-1d1c-4cd9-81a3-48266b8620f7 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:51.205201 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b7392-1d1c-4cd9-81a3-48266b8620f7 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:51.205473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b7392-1d1c-4cd9-81a3-48266b8620f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:51.235863 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-218b7392-1d1c-4cd9-81a3-48266b8620f7 None None] MFA Rules not processed for user `583b7ec1b1af4d8d8709fcc017b2d153`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:51.237235 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-218b7392-1d1c-4cd9-81a3-48266b8620f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:51.310035 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1304/2586] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:51.315340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627e1af-2e98-4d3f-8acc-b8ff56c3f5f6 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:51.345968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:51.387423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627e1af-2e98-4d3f-8acc-b8ff56c3f5f6 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:51.392753 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2c212-d820-40a4-ade5-bdbcd2f53adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:51.393527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2c212-d820-40a4-ade5-bdbcd2f53adc admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:51.393898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2c212-d820-40a4-ade5-bdbcd2f53adc admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:51.394245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2c212-d820-40a4-ade5-bdbcd2f53adc admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:51.395037 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2c212-d820-40a4-ade5-bdbcd2f53adc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:51.398308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2c212-d820-40a4-ade5-bdbcd2f53adc admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:51.438668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1283/2587] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:51 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:51.441475 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b7392-1d1c-4cd9-81a3-48266b8620f7 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:51.456514 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b7392-1d1c-4cd9-81a3-48266b8620f7 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:51.501646 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:51.518747 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826305cc-8e3f-4ee6-8115-ffc59a659373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:51.519366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826305cc-8e3f-4ee6-8115-ffc59a659373 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:51.519664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826305cc-8e3f-4ee6-8115-ffc59a659373 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:51.519911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826305cc-8e3f-4ee6-8115-ffc59a659373 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:51.520571 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826305cc-8e3f-4ee6-8115-ffc59a659373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:51.522510 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826305cc-8e3f-4ee6-8115-ffc59a659373 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:51.544707 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1305/2588] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:51 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:51.548558 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2c212-d820-40a4-ade5-bdbcd2f53adc admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:51.560643 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2c212-d820-40a4-ade5-bdbcd2f53adc admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:51.565265 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dbca3-decd-4cfb-b1c7-fff2351f939c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:51.565989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dbca3-decd-4cfb-b1c7-fff2351f939c admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:51.566331 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dbca3-decd-4cfb-b1c7-fff2351f939c admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:51.566647 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dbca3-decd-4cfb-b1c7-fff2351f939c admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:51.567334 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dbca3-decd-4cfb-b1c7-fff2351f939c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:51.569720 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dbca3-decd-4cfb-b1c7-fff2351f939c admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:51.580986 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1284/2589] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:51.585271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826305cc-8e3f-4ee6-8115-ffc59a659373 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:51.595430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826305cc-8e3f-4ee6-8115-ffc59a659373 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:51.600108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5e3d0-b131-471e-9a5b-dab3dc6355d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:51.600764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5e3d0-b131-471e-9a5b-dab3dc6355d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:51.601044 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5e3d0-b131-471e-9a5b-dab3dc6355d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:51.601319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5e3d0-b131-471e-9a5b-dab3dc6355d1 admin admin] PATH_INFO: `/v3/projects/a74be84cd96b45fd8f461e82014ed6e9/users/7845a1382d6a49cfbd73378f0ff072da/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:51.617693 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5e3d0-b131-471e-9a5b-dab3dc6355d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a74be84cd96b45fd8f461e82014ed6e9, user_id=7845a1382d6a49cfbd73378f0ff072da, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:51.620473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5e3d0-b131-471e-9a5b-dab3dc6355d1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:51.635681 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1306/2590] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:51 2022] PUT /identity/v3/projects/a74be84cd96b45fd8f461e82014ed6e9/users/7845a1382d6a49cfbd73378f0ff072da/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:51.643144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dbca3-decd-4cfb-b1c7-fff2351f939c admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:51.654561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dbca3-decd-4cfb-b1c7-fff2351f939c admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:51.717747 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c38ab-e011-41ee-9698-3a337c36b224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:51.718363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c38ab-e011-41ee-9698-3a337c36b224 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:51.718663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c38ab-e011-41ee-9698-3a337c36b224 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:51.718937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c38ab-e011-41ee-9698-3a337c36b224 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:51.719655 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c38ab-e011-41ee-9698-3a337c36b224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:51.721243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c38ab-e011-41ee-9698-3a337c36b224 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:51.735644 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1285/2591] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:51.737121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5e3d0-b131-471e-9a5b-dab3dc6355d1 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:51.748027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5e3d0-b131-471e-9a5b-dab3dc6355d1 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:51.752945 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca5973-d8c4-45fc-9694-d407c18c49fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:51.753729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca5973-d8c4-45fc-9694-d407c18c49fd admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:51.754064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca5973-d8c4-45fc-9694-d407c18c49fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:51.754400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca5973-d8c4-45fc-9694-d407c18c49fd admin admin] PATH_INFO: `/v3/projects/a74be84cd96b45fd8f461e82014ed6e9/users/7845a1382d6a49cfbd73378f0ff072da/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:51.756636 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca5973-d8c4-45fc-9694-d407c18c49fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a74be84cd96b45fd8f461e82014ed6e9, user_id=7845a1382d6a49cfbd73378f0ff072da, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:51.759414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca5973-d8c4-45fc-9694-d407c18c49fd admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:51.772620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1307/2592] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:51 2022] PUT /identity/v3/projects/a74be84cd96b45fd8f461e82014ed6e9/users/7845a1382d6a49cfbd73378f0ff072da/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:51.783802 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7dbb-f3dd-4ca3-8f95-7c08af2ce412 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:51.784107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7dbb-f3dd-4ca3-8f95-7c08af2ce412 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:51.784518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7dbb-f3dd-4ca3-8f95-7c08af2ce412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:51.816418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-6d1f7dbb-f3dd-4ca3-8f95-7c08af2ce412 None None] MFA Rules not processed for user `7845a1382d6a49cfbd73378f0ff072da`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:51.818169 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6d1f7dbb-f3dd-4ca3-8f95-7c08af2ce412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:51.894752 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1286/2593] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:51.906028 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca5973-d8c4-45fc-9694-d407c18c49fd admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:51.908038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-adca5973-d8c4-45fc-9694-d407c18c49fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:51.965965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca5973-d8c4-45fc-9694-d407c18c49fd admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:51.969583 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98911a-73e9-44f6-95c9-f563e8671875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:51.970191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98911a-73e9-44f6-95c9-f563e8671875 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:51.970468 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98911a-73e9-44f6-95c9-f563e8671875 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:51.970769 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98911a-73e9-44f6-95c9-f563e8671875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:51.982178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98911a-73e9-44f6-95c9-f563e8671875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:51.983622 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98911a-73e9-44f6-95c9-f563e8671875 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:51.994485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1308/2594] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:52.089165 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f7dbb-f3dd-4ca3-8f95-7c08af2ce412 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:52.090803 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6d1f7dbb-f3dd-4ca3-8f95-7c08af2ce412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:52.121045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:52.155335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f7dbb-f3dd-4ca3-8f95-7c08af2ce412 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:52.159073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691ef93-e465-4254-9094-2f010541644b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:52.159714 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691ef93-e465-4254-9094-2f010541644b service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:52.159994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691ef93-e465-4254-9094-2f010541644b service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:52.160398 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691ef93-e465-4254-9094-2f010541644b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:52.162389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9691ef93-e465-4254-9094-2f010541644b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:52.234531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691ef93-e465-4254-9094-2f010541644b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:52.236193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691ef93-e465-4254-9094-2f010541644b service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:52.247432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1287/2595] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:52.575553 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98911a-73e9-44f6-95c9-f563e8671875 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:52.576980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8b98911a-73e9-44f6-95c9-f563e8671875 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:52.589805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98911a-73e9-44f6-95c9-f563e8671875 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:52.593717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68bc21-a554-43bf-b0b9-8ee7a136e176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:52.594356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68bc21-a554-43bf-b0b9-8ee7a136e176 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:52.594690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68bc21-a554-43bf-b0b9-8ee7a136e176 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:52.594954 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68bc21-a554-43bf-b0b9-8ee7a136e176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:52.660791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68bc21-a554-43bf-b0b9-8ee7a136e176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:52.662932 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68bc21-a554-43bf-b0b9-8ee7a136e176 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:52.675854 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1309/2596] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:55.728192 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691ef93-e465-4254-9094-2f010541644b service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:55.729637 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9691ef93-e465-4254-9094-2f010541644b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:55.743325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691ef93-e465-4254-9094-2f010541644b service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:55.750059 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8adbe-d14e-4752-8c4e-aefdca948055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:55.750764 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8adbe-d14e-4752-8c4e-aefdca948055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:55.751045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8adbe-d14e-4752-8c4e-aefdca948055 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:55.751321 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8adbe-d14e-4752-8c4e-aefdca948055 admin admin] PATH_INFO: `/v3/users/e8f7435928a74258af5478959bd939bb` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:55.752496 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8adbe-d14e-4752-8c4e-aefdca948055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f7435928a74258af5478959bd939bb)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:55.754443 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8adbe-d14e-4752-8c4e-aefdca948055 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:55.803570 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2da8adbe-d14e-4752-8c4e-aefdca948055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f7435928a74258af5478959bd939bb'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:55.811161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2da8adbe-d14e-4752-8c4e-aefdca948055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f7435928a74258af5478959bd939bb'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:55.811863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2da8adbe-d14e-4752-8c4e-aefdca948055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f7435928a74258af5478959bd939bb'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:55.824620 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2da8adbe-d14e-4752-8c4e-aefdca948055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f7435928a74258af5478959bd939bb'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:55.831950 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2da8adbe-d14e-4752-8c4e-aefdca948055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f7435928a74258af5478959bd939bb'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:55.838595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1288/2597] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:57:55 2022] DELETE /identity/v3/users/e8f7435928a74258af5478959bd939bb => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:56.025324 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68bc21-a554-43bf-b0b9-8ee7a136e176 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:56.040052 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68bc21-a554-43bf-b0b9-8ee7a136e176 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:56.076366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:56.105858 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89efa2-f487-4fda-a154-f2332ba57e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:56.106493 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89efa2-f487-4fda-a154-f2332ba57e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:56.106799 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89efa2-f487-4fda-a154-f2332ba57e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:56.107068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89efa2-f487-4fda-a154-f2332ba57e53 admin admin] PATH_INFO: `/v3/projects/9be735c81c7a41b8a482e8f72a71bdef` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:56.108208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89efa2-f487-4fda-a154-f2332ba57e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9be735c81c7a41b8a482e8f72a71bdef)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:56.110243 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89efa2-f487-4fda-a154-f2332ba57e53 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:56.165601 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fc89efa2-f487-4fda-a154-f2332ba57e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9be735c81c7a41b8a482e8f72a71bdef', 'request_id': 'req-fc89efa2-f487-4fda-a154-f2332ba57e53'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:56.172160 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fc89efa2-f487-4fda-a154-f2332ba57e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9be735c81c7a41b8a482e8f72a71bdef', 'request_id': 'req-fc89efa2-f487-4fda-a154-f2332ba57e53'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:56.180634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fc89efa2-f487-4fda-a154-f2332ba57e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9be735c81c7a41b8a482e8f72a71bdef', 'request_id': 'req-fc89efa2-f487-4fda-a154-f2332ba57e53'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:56.191918 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fc89efa2-f487-4fda-a154-f2332ba57e53 admin admin] The token cache is being invalidate because project 9be735c81c7a41b8a482e8f72a71bdef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:57:56.192307 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fc89efa2-f487-4fda-a154-f2332ba57e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:56.193086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fc89efa2-f487-4fda-a154-f2332ba57e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:57:56.194610 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1310/2598] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:57:56 2022] DELETE /identity/v3/projects/9be735c81c7a41b8a482e8f72a71bdef => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:57:56.199638 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae422b7-60ef-43f0-b559-0b32e212dcd4 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:56.199942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae422b7-60ef-43f0-b559-0b32e212dcd4 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:56.200239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae422b7-60ef-43f0-b559-0b32e212dcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:56.223337 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-6ae422b7-60ef-43f0-b559-0b32e212dcd4 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:56.225415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6ae422b7-60ef-43f0-b559-0b32e212dcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:56.269077 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:56.290064 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1289/2599] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:56.298106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56b5ff-02dd-4734-9f29-dbeceaa80e3c None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:56.298391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56b5ff-02dd-4734-9f29-dbeceaa80e3c None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:56.298692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56b5ff-02dd-4734-9f29-dbeceaa80e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:56.318898 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-da56b5ff-02dd-4734-9f29-dbeceaa80e3c None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:56.320123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-da56b5ff-02dd-4734-9f29-dbeceaa80e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:56.327944 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1311/2600] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:56.332287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae422b7-60ef-43f0-b559-0b32e212dcd4 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:56.343115 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae422b7-60ef-43f0-b559-0b32e212dcd4 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:56.347748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cfbce-1db9-47e7-97a7-eaefacadb02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:56.348356 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cfbce-1db9-47e7-97a7-eaefacadb02a admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:56.348637 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cfbce-1db9-47e7-97a7-eaefacadb02a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:56.348937 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cfbce-1db9-47e7-97a7-eaefacadb02a admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:56.350350 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98cfbce-1db9-47e7-97a7-eaefacadb02a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:57:56.350675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98cfbce-1db9-47e7-97a7-eaefacadb02a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:56.352267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98cfbce-1db9-47e7-97a7-eaefacadb02a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:56.363921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1290/2601] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:57:56.369078 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56b5ff-02dd-4734-9f29-dbeceaa80e3c None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:56.378238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56b5ff-02dd-4734-9f29-dbeceaa80e3c None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:56.383472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970fbc3-3572-4878-9c55-fe7ba7d534c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:56.384086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970fbc3-3572-4878-9c55-fe7ba7d534c8 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:56.384364 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970fbc3-3572-4878-9c55-fe7ba7d534c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:56.384658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970fbc3-3572-4878-9c55-fe7ba7d534c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:56.385334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970fbc3-3572-4878-9c55-fe7ba7d534c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:56.387303 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970fbc3-3572-4878-9c55-fe7ba7d534c8 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:56.415238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1312/2602] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:56 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:56.419502 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cfbce-1db9-47e7-97a7-eaefacadb02a admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:56.430871 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cfbce-1db9-47e7-97a7-eaefacadb02a admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:56.491111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42eb30a-da09-4bd5-8601-0a6942fb74b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:56.491735 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42eb30a-da09-4bd5-8601-0a6942fb74b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:56.492016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42eb30a-da09-4bd5-8601-0a6942fb74b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:56.492303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42eb30a-da09-4bd5-8601-0a6942fb74b5 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:56.492978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42eb30a-da09-4bd5-8601-0a6942fb74b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:56.494981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42eb30a-da09-4bd5-8601-0a6942fb74b5 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:56.521519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1291/2603] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:56 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:56.522989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970fbc3-3572-4878-9c55-fe7ba7d534c8 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:56.533882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970fbc3-3572-4878-9c55-fe7ba7d534c8 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:56.539376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6b3a0-306b-488f-9509-67c464725137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:56.540041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6b3a0-306b-488f-9509-67c464725137 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:56.540323 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6b3a0-306b-488f-9509-67c464725137 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:56.540598 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6b3a0-306b-488f-9509-67c464725137 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:56.541366 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6b3a0-306b-488f-9509-67c464725137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:56.542914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6b3a0-306b-488f-9509-67c464725137 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:56.554407 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1313/2604] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:56.558500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42eb30a-da09-4bd5-8601-0a6942fb74b5 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:56.568957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42eb30a-da09-4bd5-8601-0a6942fb74b5 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:56.573758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed2837-371b-4c9a-b16d-252e2a53879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:56.574374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed2837-371b-4c9a-b16d-252e2a53879b admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:56.574707 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed2837-371b-4c9a-b16d-252e2a53879b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:56.574969 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed2837-371b-4c9a-b16d-252e2a53879b admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:56.575690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed2837-371b-4c9a-b16d-252e2a53879b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:56.577291 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed2837-371b-4c9a-b16d-252e2a53879b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:56.588418 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1292/2605] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:56.592695 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6b3a0-306b-488f-9509-67c464725137 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:56.603540 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6b3a0-306b-488f-9509-67c464725137 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:56.608329 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce158a2-0ec4-4088-8557-5675adf908f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:56.608999 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce158a2-0ec4-4088-8557-5675adf908f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:56.609275 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce158a2-0ec4-4088-8557-5675adf908f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:56.609547 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce158a2-0ec4-4088-8557-5675adf908f0 admin admin] PATH_INFO: `/v3/projects/13ab4ff89fac4c43883f9103339229ed/users/35f4587abb5c46ab96ac2e8b600ce241/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:56.630613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce158a2-0ec4-4088-8557-5675adf908f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ab4ff89fac4c43883f9103339229ed, user_id=35f4587abb5c46ab96ac2e8b600ce241, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:56.633359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce158a2-0ec4-4088-8557-5675adf908f0 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:56.647208 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1314/2606] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:56 2022] PUT /identity/v3/projects/13ab4ff89fac4c43883f9103339229ed/users/35f4587abb5c46ab96ac2e8b600ce241/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:57:56.652255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59c82-01f4-4984-92eb-46f28f02d972 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:56.652713 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59c82-01f4-4984-92eb-46f28f02d972 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:56.652990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59c82-01f4-4984-92eb-46f28f02d972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:56.673167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-64c59c82-01f4-4984-92eb-46f28f02d972 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:56.674360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-64c59c82-01f4-4984-92eb-46f28f02d972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:56.738160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1293/2607] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:56.748380 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce158a2-0ec4-4088-8557-5675adf908f0 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:56.751048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5ce158a2-0ec4-4088-8557-5675adf908f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:56.886673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:56.897880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce158a2-0ec4-4088-8557-5675adf908f0 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:56.902568 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30644c90-aef0-4fb9-b8b9-bbe8f326be60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:56.903391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30644c90-aef0-4fb9-b8b9-bbe8f326be60 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:56.903772 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30644c90-aef0-4fb9-b8b9-bbe8f326be60 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:56.904167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30644c90-aef0-4fb9-b8b9-bbe8f326be60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:56.921042 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30644c90-aef0-4fb9-b8b9-bbe8f326be60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:56.922687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30644c90-aef0-4fb9-b8b9-bbe8f326be60 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:56.935392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1315/2608] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:56.985516 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59c82-01f4-4984-92eb-46f28f02d972 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:56.986935 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-64c59c82-01f4-4984-92eb-46f28f02d972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:57.064941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59c82-01f4-4984-92eb-46f28f02d972 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:57.068907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e14871-5575-4443-80c7-856af793cfae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:57.069632 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e14871-5575-4443-80c7-856af793cfae service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:57.069836 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e14871-5575-4443-80c7-856af793cfae service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:57.070101 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e14871-5575-4443-80c7-856af793cfae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:57.072251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b4e14871-5575-4443-80c7-856af793cfae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:57.077412 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:57.144428 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e14871-5575-4443-80c7-856af793cfae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:57.146201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e14871-5575-4443-80c7-856af793cfae service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:57.158123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1294/2609] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:57:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:57:57.736777 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d31fef-f431-474a-a716-23945efd33f2 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:57.737331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d31fef-f431-474a-a716-23945efd33f2 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:57.737331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d31fef-f431-474a-a716-23945efd33f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:57.771888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c0d31fef-f431-474a-a716-23945efd33f2 None None] MFA Rules not processed for user `35f4587abb5c46ab96ac2e8b600ce241`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:57.773289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c0d31fef-f431-474a-a716-23945efd33f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:57.862315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1316/2610] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:57 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:57.867705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e14871-5575-4443-80c7-856af793cfae service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:57.886218 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e14871-5575-4443-80c7-856af793cfae service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:57.891582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293825f-5c51-4092-ba9b-9db46556778b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:57.892197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293825f-5c51-4092-ba9b-9db46556778b admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:57.892476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293825f-5c51-4092-ba9b-9db46556778b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:57.893087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293825f-5c51-4092-ba9b-9db46556778b admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:57.893536 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293825f-5c51-4092-ba9b-9db46556778b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:57.895768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293825f-5c51-4092-ba9b-9db46556778b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:57.930716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1295/2611] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:57:57 2022] POST /identity/v3/projects => generated 403 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:57:57.931948 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d31fef-f431-474a-a716-23945efd33f2 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:57.944806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d31fef-f431-474a-a716-23945efd33f2 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:58.001527 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ef9f0-798b-4179-9e14-711681c4d388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:58.002153 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ef9f0-798b-4179-9e14-711681c4d388 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:58.002430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ef9f0-798b-4179-9e14-711681c4d388 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:58.002736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ef9f0-798b-4179-9e14-711681c4d388 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:58.003395 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ef9f0-798b-4179-9e14-711681c4d388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:58.005820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ef9f0-798b-4179-9e14-711681c4d388 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:58.029592 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1317/2612] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:57:57 2022] POST /identity/v3/users => generated 308 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:57:58.034670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293825f-5c51-4092-ba9b-9db46556778b admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:58.048799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293825f-5c51-4092-ba9b-9db46556778b admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:58.057385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea7296-0d16-4bc8-b04f-3d1c72dde5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:58.058024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea7296-0d16-4bc8-b04f-3d1c72dde5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:58.058305 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea7296-0d16-4bc8-b04f-3d1c72dde5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:58.058576 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea7296-0d16-4bc8-b04f-3d1c72dde5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:58.059302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea7296-0d16-4bc8-b04f-3d1c72dde5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:58.060970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea7296-0d16-4bc8-b04f-3d1c72dde5a7 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:58.074683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1296/2613] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:57:58.078815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ef9f0-798b-4179-9e14-711681c4d388 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:58.090357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ef9f0-798b-4179-9e14-711681c4d388 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:58.095332 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b708d-bc18-4190-9802-e3259adf0ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:58.095995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b708d-bc18-4190-9802-e3259adf0ae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:58.096277 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b708d-bc18-4190-9802-e3259adf0ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:58.096546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b708d-bc18-4190-9802-e3259adf0ae3 admin admin] PATH_INFO: `/v3/projects/303291b07436490f81984e21394f0d28/users/82ba04cc86e744d5a61c9eac140db6f1/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:58.113333 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b708d-bc18-4190-9802-e3259adf0ae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=303291b07436490f81984e21394f0d28, user_id=82ba04cc86e744d5a61c9eac140db6f1, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:58.116059 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b708d-bc18-4190-9802-e3259adf0ae3 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:58.133447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1318/2614] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:58 2022] PUT /identity/v3/projects/303291b07436490f81984e21394f0d28/users/82ba04cc86e744d5a61c9eac140db6f1/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:57:58.137767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea7296-0d16-4bc8-b04f-3d1c72dde5a7 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:58.149056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea7296-0d16-4bc8-b04f-3d1c72dde5a7 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:58.208040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95118e46-deed-4fc7-a417-a659b5cb6068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:58.209044 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95118e46-deed-4fc7-a417-a659b5cb6068 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:58.209326 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95118e46-deed-4fc7-a417-a659b5cb6068 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:58.209600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95118e46-deed-4fc7-a417-a659b5cb6068 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:58.210295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95118e46-deed-4fc7-a417-a659b5cb6068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:58.211980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95118e46-deed-4fc7-a417-a659b5cb6068 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:58.222393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1297/2615] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:57:58.227092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b708d-bc18-4190-9802-e3259adf0ae3 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:58.241079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b708d-bc18-4190-9802-e3259adf0ae3 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:58.246049 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3a5f5-a049-446b-b99f-0cedcf8343aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:58.246908 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3a5f5-a049-446b-b99f-0cedcf8343aa admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:58.247190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3a5f5-a049-446b-b99f-0cedcf8343aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:58.247463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3a5f5-a049-446b-b99f-0cedcf8343aa admin admin] PATH_INFO: `/v3/projects/303291b07436490f81984e21394f0d28/users/82ba04cc86e744d5a61c9eac140db6f1/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:58.249444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3a5f5-a049-446b-b99f-0cedcf8343aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=303291b07436490f81984e21394f0d28, user_id=82ba04cc86e744d5a61c9eac140db6f1, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:58.252115 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3a5f5-a049-446b-b99f-0cedcf8343aa admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:57:58.264895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1319/2616] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:57:58 2022] PUT /identity/v3/projects/303291b07436490f81984e21394f0d28/users/82ba04cc86e744d5a61c9eac140db6f1/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:57:59.531844 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c36621-e292-43bf-918b-abb5824b6c57 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:59.532748 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c36621-e292-43bf-918b-abb5824b6c57 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:59.533217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c36621-e292-43bf-918b-abb5824b6c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:59.636209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-66c36621-e292-43bf-918b-abb5824b6c57 None None] MFA Rules not processed for user `82ba04cc86e744d5a61c9eac140db6f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:57:59.639956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-66c36621-e292-43bf-918b-abb5824b6c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:59.781791 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3a5f5-a049-446b-b99f-0cedcf8343aa admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:57:59.784092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-9ee3a5f5-a049-446b-b99f-0cedcf8343aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:57:59.809882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:59.837917 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1298/2617] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:57:59 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:57:59.871068 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:57:59.937721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3a5f5-a049-446b-b99f-0cedcf8343aa admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:57:59.950832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9113d0b-8705-4003-b55d-04aa3225e732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:57:59.951773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9113d0b-8705-4003-b55d-04aa3225e732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:57:59.952262 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9113d0b-8705-4003-b55d-04aa3225e732 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:57:59.952728 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9113d0b-8705-4003-b55d-04aa3225e732 admin admin] PATH_INFO: `/v3/users/c21891153f6141e09f39ae05a3707eb7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:57:59.955110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9113d0b-8705-4003-b55d-04aa3225e732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c21891153f6141e09f39ae05a3707eb7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:57:59.959723 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9113d0b-8705-4003-b55d-04aa3225e732 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:00.045548 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b9113d0b-8705-4003-b55d-04aa3225e732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c21891153f6141e09f39ae05a3707eb7'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:00.074883 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b9113d0b-8705-4003-b55d-04aa3225e732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c21891153f6141e09f39ae05a3707eb7'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:00.114426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b9113d0b-8705-4003-b55d-04aa3225e732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c21891153f6141e09f39ae05a3707eb7'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:00.122700 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b9113d0b-8705-4003-b55d-04aa3225e732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c21891153f6141e09f39ae05a3707eb7'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:00.123533 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-b9113d0b-8705-4003-b55d-04aa3225e732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c21891153f6141e09f39ae05a3707eb7'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:00.150587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c36621-e292-43bf-918b-abb5824b6c57 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:00.152414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-66c36621-e292-43bf-918b-abb5824b6c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:00.157567 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1320/2618] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:57:59 2022] DELETE /identity/v3/users/c21891153f6141e09f39ae05a3707eb7 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:58:00.164445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1bde5-8c19-4ee1-90c6-592406b11bdb None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:00.165677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1bde5-8c19-4ee1-90c6-592406b11bdb None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:00.168320 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1bde5-8c19-4ee1-90c6-592406b11bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:00.201843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-d9b1bde5-8c19-4ee1-90c6-592406b11bdb None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:00.213192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d9b1bde5-8c19-4ee1-90c6-592406b11bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:00.235031 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c36621-e292-43bf-918b-abb5824b6c57 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:00.314148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1321/2619] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:00.325075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1bde5-8c19-4ee1-90c6-592406b11bdb None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:00.326574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d9b1bde5-8c19-4ee1-90c6-592406b11bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:00.341450 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631f706-77d2-4407-a7bf-b787f47ea4ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:00.342209 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631f706-77d2-4407-a7bf-b787f47ea4ae service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:00.342661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631f706-77d2-4407-a7bf-b787f47ea4ae service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:00.343040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631f706-77d2-4407-a7bf-b787f47ea4ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:00.345129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9631f706-77d2-4407-a7bf-b787f47ea4ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:00.385051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:00.392921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1bde5-8c19-4ee1-90c6-592406b11bdb None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:00.398907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c797570-349d-476f-b0eb-7a81a24c6355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:00.399840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c797570-349d-476f-b0eb-7a81a24c6355 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:00.400414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c797570-349d-476f-b0eb-7a81a24c6355 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:00.400738 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c797570-349d-476f-b0eb-7a81a24c6355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:00.410933 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c797570-349d-476f-b0eb-7a81a24c6355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:00.412436 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c797570-349d-476f-b0eb-7a81a24c6355 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:00.421860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631f706-77d2-4407-a7bf-b787f47ea4ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:00.423449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631f706-77d2-4407-a7bf-b787f47ea4ae service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:00.424219 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1322/2620] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:58:00.436472 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1299/2621] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:01.727514 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c797570-349d-476f-b0eb-7a81a24c6355 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:01.729485 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4c797570-349d-476f-b0eb-7a81a24c6355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:01.745261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c797570-349d-476f-b0eb-7a81a24c6355 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:01.761217 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:01.762178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:01.762655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:01.763133 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] PATH_INFO: `/v3/projects/2ec668f9c77a44d59149ed10900dfec1` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:01.764849 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ec668f9c77a44d59149ed10900dfec1)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:01.767986 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:01.859012 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ec668f9c77a44d59149ed10900dfec1', 'request_id': 'req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:01.866708 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ec668f9c77a44d59149ed10900dfec1', 'request_id': 'req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:01.876624 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ec668f9c77a44d59149ed10900dfec1', 'request_id': 'req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:01.894046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] The token cache is being invalidate because project 2ec668f9c77a44d59149ed10900dfec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:01.894469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:01.897767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:01.899832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1323/2622] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:01 2022] DELETE /identity/v3/projects/2ec668f9c77a44d59149ed10900dfec1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:01.904224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631f706-77d2-4407-a7bf-b787f47ea4ae service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:01.918076 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631f706-77d2-4407-a7bf-b787f47ea4ae service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:02.014505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:02.015204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:02.015487 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:02.015975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] PATH_INFO: `/v3/users/583b7ec1b1af4d8d8709fcc017b2d153` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:02.019599 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=583b7ec1b1af4d8d8709fcc017b2d153)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:02.022008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:02.027401 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:02.102236 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '583b7ec1b1af4d8d8709fcc017b2d153'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:02.112332 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '583b7ec1b1af4d8d8709fcc017b2d153'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:02.113331 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '583b7ec1b1af4d8d8709fcc017b2d153'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:02.129413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '583b7ec1b1af4d8d8709fcc017b2d153'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:02.144609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '583b7ec1b1af4d8d8709fcc017b2d153'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:02.152098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1300/2623] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:01 2022] DELETE /identity/v3/users/583b7ec1b1af4d8d8709fcc017b2d153 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:03.262011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:03.264438 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:03.300486 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa0d43-f7d7-40bf-9e3c-36d22336bfc0 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:03.471496 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:03.473976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:03.526184 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:03.526876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:03.527147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:03.529829 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] PATH_INFO: `/v3/projects/f52736554ec841fc8d65ee3c04bb55ac` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:03.533600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f52736554ec841fc8d65ee3c04bb55ac)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:03.535618 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:03.701926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9c02f-8fd9-4dab-8379-825a24c967cb admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:03.717226 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:03.717907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:03.718943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:03.718943 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] PATH_INFO: `/v3/users/35f4587abb5c46ab96ac2e8b600ce241` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:03.728333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f4587abb5c46ab96ac2e8b600ce241)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:03.730295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:03.760632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f52736554ec841fc8d65ee3c04bb55ac', 'request_id': 'req-4a777b6b-3a41-4d06-8d73-543ab0c7261f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:03.773615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f52736554ec841fc8d65ee3c04bb55ac', 'request_id': 'req-4a777b6b-3a41-4d06-8d73-543ab0c7261f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:03.807160 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f52736554ec841fc8d65ee3c04bb55ac', 'request_id': 'req-4a777b6b-3a41-4d06-8d73-543ab0c7261f'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:03.857637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] The token cache is being invalidate because project f52736554ec841fc8d65ee3c04bb55ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:03.860487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:03.861815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:03.870749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1324/2624] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:03 2022] DELETE /identity/v3/projects/f52736554ec841fc8d65ee3c04bb55ac => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:03.873295 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:03.878134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:03.902539 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:03.921233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f4587abb5c46ab96ac2e8b600ce241'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:03.931444 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f4587abb5c46ab96ac2e8b600ce241'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:03.932195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f4587abb5c46ab96ac2e8b600ce241'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:03.950572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f4587abb5c46ab96ac2e8b600ce241'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:03.964625 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f4587abb5c46ab96ac2e8b600ce241'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:03.967168 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:03.978427 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1301/2625] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:03 2022] DELETE /identity/v3/users/35f4587abb5c46ab96ac2e8b600ce241 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:04.070157 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a777b6b-3a41-4d06-8d73-543ab0c7261f admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:04.181889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:04.182556 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:04.182862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:04.183138 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] PATH_INFO: `/v3/users/7845a1382d6a49cfbd73378f0ff072da` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:04.207593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7845a1382d6a49cfbd73378f0ff072da)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:04.209715 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:04.311930 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7845a1382d6a49cfbd73378f0ff072da'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.339840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7845a1382d6a49cfbd73378f0ff072da'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.354855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7845a1382d6a49cfbd73378f0ff072da'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.355583 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7845a1382d6a49cfbd73378f0ff072da'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.356253 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7845a1382d6a49cfbd73378f0ff072da'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.394894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1325/2626] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:03 2022] DELETE /identity/v3/users/7845a1382d6a49cfbd73378f0ff072da => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:04.468822 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:04.470014 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:04.493351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b6574-74b8-402c-b429-8980c1f4b250 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:04.618912 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:04.619797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:04.621800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:04.622294 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] PATH_INFO: `/v3/projects/13ab4ff89fac4c43883f9103339229ed` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:04.626005 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=13ab4ff89fac4c43883f9103339229ed)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:04.632228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:04.709434 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13ab4ff89fac4c43883f9103339229ed', 'request_id': 'req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.717430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13ab4ff89fac4c43883f9103339229ed', 'request_id': 'req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.727866 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13ab4ff89fac4c43883f9103339229ed', 'request_id': 'req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.739606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] The token cache is being invalidate because project 13ab4ff89fac4c43883f9103339229ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:04.740142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.741509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.746719 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:04.747269 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1302/2627] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:04 2022] DELETE /identity/v3/projects/13ab4ff89fac4c43883f9103339229ed => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:04.748203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:04.765170 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc272698-b404-4dcc-998a-e66dd0379949 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:04.808464 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:04.858274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:04.859090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:04.860326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:04.860326 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] PATH_INFO: `/v3/users/82ba04cc86e744d5a61c9eac140db6f1` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:04.861480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82ba04cc86e744d5a61c9eac140db6f1)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:04.864291 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:04.942324 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82ba04cc86e744d5a61c9eac140db6f1'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.959121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82ba04cc86e744d5a61c9eac140db6f1'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.967890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82ba04cc86e744d5a61c9eac140db6f1'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.968642 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82ba04cc86e744d5a61c9eac140db6f1'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:04.969363 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82ba04cc86e744d5a61c9eac140db6f1'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:05.015956 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1326/2628] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:04 2022] DELETE /identity/v3/users/82ba04cc86e744d5a61c9eac140db6f1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:05.319158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:05.321690 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:05.337431 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7df0f7-06a0-4227-98a2-5ec40bafb1aa admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:05.452029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dccc46-6b02-4f71-9723-439ebd65d1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:05.452682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dccc46-6b02-4f71-9723-439ebd65d1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:05.453028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dccc46-6b02-4f71-9723-439ebd65d1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:05.453309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dccc46-6b02-4f71-9723-439ebd65d1eb admin admin] PATH_INFO: `/v3/projects/a74be84cd96b45fd8f461e82014ed6e9` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:05.454517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dccc46-6b02-4f71-9723-439ebd65d1eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a74be84cd96b45fd8f461e82014ed6e9)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:05.456528 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dccc46-6b02-4f71-9723-439ebd65d1eb admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:05.486294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:05.488017 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:05.504987 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddde740-f597-4432-aacf-ee89b6fbd040 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:05.512472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9fd32-553a-4237-bddd-f452c5139e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:05.513373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9fd32-553a-4237-bddd-f452c5139e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:05.513927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9fd32-553a-4237-bddd-f452c5139e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:05.514201 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9fd32-553a-4237-bddd-f452c5139e5d admin admin] PATH_INFO: `/v3/projects/303291b07436490f81984e21394f0d28` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:05.515593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9fd32-553a-4237-bddd-f452c5139e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=303291b07436490f81984e21394f0d28)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:05.517763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9fd32-553a-4237-bddd-f452c5139e5d admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:05.532166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:05.556541 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-13dccc46-6b02-4f71-9723-439ebd65d1eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a74be84cd96b45fd8f461e82014ed6e9', 'request_id': 'req-13dccc46-6b02-4f71-9723-439ebd65d1eb'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:05.563087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-13dccc46-6b02-4f71-9723-439ebd65d1eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a74be84cd96b45fd8f461e82014ed6e9', 'request_id': 'req-13dccc46-6b02-4f71-9723-439ebd65d1eb'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:05.573470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-13dccc46-6b02-4f71-9723-439ebd65d1eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a74be84cd96b45fd8f461e82014ed6e9', 'request_id': 'req-13dccc46-6b02-4f71-9723-439ebd65d1eb'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:05.577275 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-58e9fd32-553a-4237-bddd-f452c5139e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '303291b07436490f81984e21394f0d28', 'request_id': 'req-58e9fd32-553a-4237-bddd-f452c5139e5d'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:05.585553 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-58e9fd32-553a-4237-bddd-f452c5139e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '303291b07436490f81984e21394f0d28', 'request_id': 'req-58e9fd32-553a-4237-bddd-f452c5139e5d'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:05.588001 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-13dccc46-6b02-4f71-9723-439ebd65d1eb admin admin] The token cache is being invalidate because project a74be84cd96b45fd8f461e82014ed6e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:05.588365 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-13dccc46-6b02-4f71-9723-439ebd65d1eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:05.589077 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-13dccc46-6b02-4f71-9723-439ebd65d1eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:05.590705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1303/2629] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:05 2022] DELETE /identity/v3/projects/a74be84cd96b45fd8f461e82014ed6e9 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:05.595398 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-58e9fd32-553a-4237-bddd-f452c5139e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '303291b07436490f81984e21394f0d28', 'request_id': 'req-58e9fd32-553a-4237-bddd-f452c5139e5d'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:05.596560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c23a0-b243-45e3-8979-43abd94e1c4d None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:05.596847 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c23a0-b243-45e3-8979-43abd94e1c4d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:05.597121 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c23a0-b243-45e3-8979-43abd94e1c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:05.607319 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-58e9fd32-553a-4237-bddd-f452c5139e5d admin admin] The token cache is being invalidate because project 303291b07436490f81984e21394f0d28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:05.607674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-58e9fd32-553a-4237-bddd-f452c5139e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:05.608394 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-58e9fd32-553a-4237-bddd-f452c5139e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:05.609964 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1327/2630] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:05 2022] DELETE /identity/v3/projects/303291b07436490f81984e21394f0d28 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:05.615127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5a67a-9c68-49c3-823b-2cff8ba2c621 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:05.615360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5a67a-9c68-49c3-823b-2cff8ba2c621 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:05.615727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5a67a-9c68-49c3-823b-2cff8ba2c621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:05.619857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-556c23a0-b243-45e3-8979-43abd94e1c4d None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:05.621257 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-556c23a0-b243-45e3-8979-43abd94e1c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:05.637910 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-e2c5a67a-9c68-49c3-823b-2cff8ba2c621 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:05.640027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-e2c5a67a-9c68-49c3-823b-2cff8ba2c621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:05.668333 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:05.722944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1304/2631] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:58:05.727638 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1328/2632] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:05.730461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d0c64-7852-4d50-8426-dbd3c1149e01 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:05.730806 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d0c64-7852-4d50-8426-dbd3c1149e01 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:05.731074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d0c64-7852-4d50-8426-dbd3c1149e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:05.735718 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc83a4f-773c-4030-b7eb-e99afb9ab375 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:05.736295 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc83a4f-773c-4030-b7eb-e99afb9ab375 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:05.736676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc83a4f-773c-4030-b7eb-e99afb9ab375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:05.754787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-4b8d0c64-7852-4d50-8426-dbd3c1149e01 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:05.757571 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-4b8d0c64-7852-4d50-8426-dbd3c1149e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:05.762123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-bfc83a4f-773c-4030-b7eb-e99afb9ab375 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:05.763745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-bfc83a4f-773c-4030-b7eb-e99afb9ab375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:05.772495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1305/2633] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:05.773133 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1329/2634] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:58:05.776640 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d0c64-7852-4d50-8426-dbd3c1149e01 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:05.778751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc83a4f-773c-4030-b7eb-e99afb9ab375 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:05.786540 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d0c64-7852-4d50-8426-dbd3c1149e01 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:05.789357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc83a4f-773c-4030-b7eb-e99afb9ab375 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:05.792369 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f3ff7-017d-417d-bdc6-00cb38082248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:05.792615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f3ff7-017d-417d-bdc6-00cb38082248 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:05.792859 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f3ff7-017d-417d-bdc6-00cb38082248 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:05.793560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f3ff7-017d-417d-bdc6-00cb38082248 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:05.794268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f3ff7-017d-417d-bdc6-00cb38082248 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:58:05.794573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f3ff7-017d-417d-bdc6-00cb38082248 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:05.794714 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b107aa2-1d14-4d23-983d-f953bafebb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:05.795371 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b107aa2-1d14-4d23-983d-f953bafebb5b admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:05.795725 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b107aa2-1d14-4d23-983d-f953bafebb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:05.796073 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b107aa2-1d14-4d23-983d-f953bafebb5b admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:05.796842 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f3ff7-017d-417d-bdc6-00cb38082248 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:05.796927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b107aa2-1d14-4d23-983d-f953bafebb5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:58:05.797293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b107aa2-1d14-4d23-983d-f953bafebb5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:05.799064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b107aa2-1d14-4d23-983d-f953bafebb5b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:05.810770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1306/2635] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:58:05.816043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1330/2636] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:58:05.816309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f3ff7-017d-417d-bdc6-00cb38082248 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:05.817067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b107aa2-1d14-4d23-983d-f953bafebb5b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:05.826989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f3ff7-017d-417d-bdc6-00cb38082248 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:05.827917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b107aa2-1d14-4d23-983d-f953bafebb5b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:05.832569 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e618cf8-bbe8-4660-8292-2f50c99a7168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:05.832725 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbe1fa-ad46-4433-9ce3-08df42891d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:05.833211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e618cf8-bbe8-4660-8292-2f50c99a7168 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:05.833493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e618cf8-bbe8-4660-8292-2f50c99a7168 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:05.833643 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbe1fa-ad46-4433-9ce3-08df42891d0b admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:05.833801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e618cf8-bbe8-4660-8292-2f50c99a7168 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:05.833938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbe1fa-ad46-4433-9ce3-08df42891d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:05.834285 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbe1fa-ad46-4433-9ce3-08df42891d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:05.834931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e618cf8-bbe8-4660-8292-2f50c99a7168 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:05.835128 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbe1fa-ad46-4433-9ce3-08df42891d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:05.836599 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e618cf8-bbe8-4660-8292-2f50c99a7168 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:05.837174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbe1fa-ad46-4433-9ce3-08df42891d0b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:05.872929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1331/2637] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:05 2022] POST /identity/v3/projects => generated 357 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:58:05.873135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1307/2638] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:05 2022] POST /identity/v3/projects => generated 401 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:05.874039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e618cf8-bbe8-4660-8292-2f50c99a7168 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:05.874194 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbe1fa-ad46-4433-9ce3-08df42891d0b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:05.966762 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e618cf8-bbe8-4660-8292-2f50c99a7168 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:05.975648 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea1d2e-fbfc-446b-8afc-5909d5093bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:05.976383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1d2e-fbfc-446b-8afc-5909d5093bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:05.976670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1d2e-fbfc-446b-8afc-5909d5093bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:05.976935 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1d2e-fbfc-446b-8afc-5909d5093bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:05.978186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea1d2e-fbfc-446b-8afc-5909d5093bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:05.980136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea1d2e-fbfc-446b-8afc-5909d5093bc1 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:05.981320 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbe1fa-ad46-4433-9ce3-08df42891d0b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:05.986126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5e210-1b01-49f6-a10c-939a18ce6044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:05.986870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5e210-1b01-49f6-a10c-939a18ce6044 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:05.987217 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5e210-1b01-49f6-a10c-939a18ce6044 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:05.987561 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5e210-1b01-49f6-a10c-939a18ce6044 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:05.988305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5e210-1b01-49f6-a10c-939a18ce6044 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:05.990649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5e210-1b01-49f6-a10c-939a18ce6044 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.010137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1308/2639] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:05 2022] POST /identity/v3/users => generated 301 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:06.014166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea1d2e-fbfc-446b-8afc-5909d5093bc1 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.016735 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1332/2640] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:05 2022] POST /identity/v3/users => generated 279 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:06.020619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5e210-1b01-49f6-a10c-939a18ce6044 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.030090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea1d2e-fbfc-446b-8afc-5909d5093bc1 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.037473 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5e210-1b01-49f6-a10c-939a18ce6044 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.038158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1138dad-733f-4c03-930c-2b44c981103d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.039071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1138dad-733f-4c03-930c-2b44c981103d admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.039416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1138dad-733f-4c03-930c-2b44c981103d admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.039751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1138dad-733f-4c03-930c-2b44c981103d admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.040512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1138dad-733f-4c03-930c-2b44c981103d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.042172 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1138dad-733f-4c03-930c-2b44c981103d admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.043806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7cbea-027a-4f81-8562-fb8c1a4915c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.044321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7cbea-027a-4f81-8562-fb8c1a4915c1 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.045038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7cbea-027a-4f81-8562-fb8c1a4915c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.045038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7cbea-027a-4f81-8562-fb8c1a4915c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.045547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:06.045668 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7cbea-027a-4f81-8562-fb8c1a4915c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.047451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7cbea-027a-4f81-8562-fb8c1a4915c1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.057019 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1309/2641] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:58:06.059632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1333/2642] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:58:06.062065 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1138dad-733f-4c03-930c-2b44c981103d admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.064762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7cbea-027a-4f81-8562-fb8c1a4915c1 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.074323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1138dad-733f-4c03-930c-2b44c981103d admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.077924 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7cbea-027a-4f81-8562-fb8c1a4915c1 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.079134 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732b86f-e329-4d05-9d3c-411c217da159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.079757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732b86f-e329-4d05-9d3c-411c217da159 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.080025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732b86f-e329-4d05-9d3c-411c217da159 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.080300 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732b86f-e329-4d05-9d3c-411c217da159 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.080992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732b86f-e329-4d05-9d3c-411c217da159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.082618 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732b86f-e329-4d05-9d3c-411c217da159 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.085666 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee369fc9-5fd8-4ce1-ad47-d465676b01d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.086670 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369fc9-5fd8-4ce1-ad47-d465676b01d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.087142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369fc9-5fd8-4ce1-ad47-d465676b01d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.087576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee369fc9-5fd8-4ce1-ad47-d465676b01d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.088550 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee369fc9-5fd8-4ce1-ad47-d465676b01d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.090501 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee369fc9-5fd8-4ce1-ad47-d465676b01d4 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.096367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1310/2643] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:06.101028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732b86f-e329-4d05-9d3c-411c217da159 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.107474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1334/2644] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:06.111528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee369fc9-5fd8-4ce1-ad47-d465676b01d4 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.116894 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732b86f-e329-4d05-9d3c-411c217da159 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.122750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51490ee4-6e23-48ef-8932-419c77941144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.123409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee369fc9-5fd8-4ce1-ad47-d465676b01d4 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.123534 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51490ee4-6e23-48ef-8932-419c77941144 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.125449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51490ee4-6e23-48ef-8932-419c77941144 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.125854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51490ee4-6e23-48ef-8932-419c77941144 admin admin] PATH_INFO: `/v3/projects/9c3575b5d053430a9167262299e25d25/users/8928c5e7521845279b971403f223885e/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.129936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399517ef-6d9b-4a10-843f-cddd3bfddc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.131461 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399517ef-6d9b-4a10-843f-cddd3bfddc10 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.131804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399517ef-6d9b-4a10-843f-cddd3bfddc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.132146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399517ef-6d9b-4a10-843f-cddd3bfddc10 admin admin] PATH_INFO: `/v3/projects/87418eff0be4435696b0b1b8c64c32ee/users/b700b468142b4ae88b17efa0112747f0/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.145385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51490ee4-6e23-48ef-8932-419c77941144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3575b5d053430a9167262299e25d25, user_id=8928c5e7521845279b971403f223885e, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.149004 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51490ee4-6e23-48ef-8932-419c77941144 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.151595 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399517ef-6d9b-4a10-843f-cddd3bfddc10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87418eff0be4435696b0b1b8c64c32ee, user_id=b700b468142b4ae88b17efa0112747f0, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.154727 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399517ef-6d9b-4a10-843f-cddd3bfddc10 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.165731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1311/2645] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:06 2022] PUT /identity/v3/projects/9c3575b5d053430a9167262299e25d25/users/8928c5e7521845279b971403f223885e/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:06.178385 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abc745-e021-4a35-b3c7-ac7571a53e7f None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.178720 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abc745-e021-4a35-b3c7-ac7571a53e7f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.178998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abc745-e021-4a35-b3c7-ac7571a53e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.182469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1335/2646] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:06 2022] PUT /identity/v3/projects/87418eff0be4435696b0b1b8c64c32ee/users/b700b468142b4ae88b17efa0112747f0/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:58:06.192733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc733d23-556e-43aa-8f3d-29da6a763b0b None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.193659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc733d23-556e-43aa-8f3d-29da6a763b0b None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.194106 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc733d23-556e-43aa-8f3d-29da6a763b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.231310 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-99abc745-e021-4a35-b3c7-ac7571a53e7f None None] MFA Rules not processed for user `8928c5e7521845279b971403f223885e`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:06.232542 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-fc733d23-556e-43aa-8f3d-29da6a763b0b None None] MFA Rules not processed for user `b700b468142b4ae88b17efa0112747f0`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:06.232671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-99abc745-e021-4a35-b3c7-ac7571a53e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:06.234226 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fc733d23-556e-43aa-8f3d-29da6a763b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:06.263880 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:06.333742 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1312/2647] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:06 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:58:06.336798 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abc745-e021-4a35-b3c7-ac7571a53e7f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.337102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1336/2648] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:06.341563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc733d23-556e-43aa-8f3d-29da6a763b0b None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.446427 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abc745-e021-4a35-b3c7-ac7571a53e7f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.451770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b9f9d-dc9a-4e02-a920-081ecabcb9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.452398 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b9f9d-dc9a-4e02-a920-081ecabcb9e4 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.452671 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b9f9d-dc9a-4e02-a920-081ecabcb9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.452941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b9f9d-dc9a-4e02-a920-081ecabcb9e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.453966 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b9f9d-dc9a-4e02-a920-081ecabcb9e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.456584 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b9f9d-dc9a-4e02-a920-081ecabcb9e4 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.460805 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc733d23-556e-43aa-8f3d-29da6a763b0b None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.466379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272f151-1ba4-489a-8793-d0fdb4412a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.467115 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272f151-1ba4-489a-8793-d0fdb4412a2f admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.467459 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272f151-1ba4-489a-8793-d0fdb4412a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.467797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272f151-1ba4-489a-8793-d0fdb4412a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.468629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272f151-1ba4-489a-8793-d0fdb4412a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.470758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272f151-1ba4-489a-8793-d0fdb4412a2f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.498946 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1313/2649] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:06 2022] POST /identity/v3/projects => generated 399 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:06.513737 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b9f9d-dc9a-4e02-a920-081ecabcb9e4 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.524430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1337/2650] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:06 2022] POST /identity/v3/projects => generated 355 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:06.528092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272f151-1ba4-489a-8793-d0fdb4412a2f admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.566422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:06.602692 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b9f9d-dc9a-4e02-a920-081ecabcb9e4 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.620756 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272f151-1ba4-489a-8793-d0fdb4412a2f admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.621051 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638518c-4387-4108-887f-7e5496fb18fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.621877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638518c-4387-4108-887f-7e5496fb18fa admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.622159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638518c-4387-4108-887f-7e5496fb18fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.622428 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638518c-4387-4108-887f-7e5496fb18fa admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.623160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638518c-4387-4108-887f-7e5496fb18fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.625088 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638518c-4387-4108-887f-7e5496fb18fa admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.634203 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22792e-3571-4dd4-a6fb-8999cb15fda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.634952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22792e-3571-4dd4-a6fb-8999cb15fda0 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.635233 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22792e-3571-4dd4-a6fb-8999cb15fda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.635512 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22792e-3571-4dd4-a6fb-8999cb15fda0 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.636358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22792e-3571-4dd4-a6fb-8999cb15fda0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.638296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22792e-3571-4dd4-a6fb-8999cb15fda0 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.649102 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1314/2651] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:06 2022] POST /identity/v3/users => generated 306 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:06.653360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638518c-4387-4108-887f-7e5496fb18fa admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.661445 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1338/2652] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:06 2022] POST /identity/v3/users => generated 284 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:06.664045 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638518c-4387-4108-887f-7e5496fb18fa admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.665091 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22792e-3571-4dd4-a6fb-8999cb15fda0 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.668819 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f45eba-4cbd-4c0a-b02a-136d343c4a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.669492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f45eba-4cbd-4c0a-b02a-136d343c4a9e admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.669786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f45eba-4cbd-4c0a-b02a-136d343c4a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.670061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f45eba-4cbd-4c0a-b02a-136d343c4a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.670966 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f45eba-4cbd-4c0a-b02a-136d343c4a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.672796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f45eba-4cbd-4c0a-b02a-136d343c4a9e admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.675148 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22792e-3571-4dd4-a6fb-8999cb15fda0 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.681510 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18fb45-c1d1-4161-9942-fb73b5a98290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.682105 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18fb45-c1d1-4161-9942-fb73b5a98290 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.682378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18fb45-c1d1-4161-9942-fb73b5a98290 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.682672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18fb45-c1d1-4161-9942-fb73b5a98290 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.683357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18fb45-c1d1-4161-9942-fb73b5a98290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.684792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1315/2653] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:58:06.684994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18fb45-c1d1-4161-9942-fb73b5a98290 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.688987 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f45eba-4cbd-4c0a-b02a-136d343c4a9e admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.696233 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1339/2654] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:06.699902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f45eba-4cbd-4c0a-b02a-136d343c4a9e admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.700421 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18fb45-c1d1-4161-9942-fb73b5a98290 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.704675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb56768-ae71-4fe0-890c-a452bc537fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.705329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb56768-ae71-4fe0-890c-a452bc537fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.705646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb56768-ae71-4fe0-890c-a452bc537fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.705878 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb56768-ae71-4fe0-890c-a452bc537fd4 admin admin] PATH_INFO: `/v3/projects/2054ad50e510497f9e10af39051d0018/users/35249cfbeb8f4945b7d7bb43bf17575e/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.710359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18fb45-c1d1-4161-9942-fb73b5a98290 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.715576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12083d69-9467-481e-8f81-ea5cfb65aa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.716331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12083d69-9467-481e-8f81-ea5cfb65aa2e admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.716723 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12083d69-9467-481e-8f81-ea5cfb65aa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.717060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12083d69-9467-481e-8f81-ea5cfb65aa2e admin admin] PATH_INFO: `/v3/projects/6ffe585843fa4042b0ecb7453eefbf14/users/0aeec5b0d57a4ac387d2d83e791e8173/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.721598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb56768-ae71-4fe0-890c-a452bc537fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2054ad50e510497f9e10af39051d0018, user_id=35249cfbeb8f4945b7d7bb43bf17575e, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.724550 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb56768-ae71-4fe0-890c-a452bc537fd4 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.732377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12083d69-9467-481e-8f81-ea5cfb65aa2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffe585843fa4042b0ecb7453eefbf14, user_id=0aeec5b0d57a4ac387d2d83e791e8173, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.735134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12083d69-9467-481e-8f81-ea5cfb65aa2e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.738033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1316/2655] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:06 2022] PUT /identity/v3/projects/2054ad50e510497f9e10af39051d0018/users/35249cfbeb8f4945b7d7bb43bf17575e/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:06.743056 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb56768-ae71-4fe0-890c-a452bc537fd4 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.749142 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1340/2656] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:06 2022] PUT /identity/v3/projects/6ffe585843fa4042b0ecb7453eefbf14/users/0aeec5b0d57a4ac387d2d83e791e8173/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:58:06.752840 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12083d69-9467-481e-8f81-ea5cfb65aa2e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.757994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:06.808124 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb56768-ae71-4fe0-890c-a452bc537fd4 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.819305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12083d69-9467-481e-8f81-ea5cfb65aa2e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.824076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a55781-6d03-4056-9984-1d50b01721a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.824691 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a55781-6d03-4056-9984-1d50b01721a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.824994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a55781-6d03-4056-9984-1d50b01721a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.825266 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a55781-6d03-4056-9984-1d50b01721a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.825962 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a55781-6d03-4056-9984-1d50b01721a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.827622 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a55781-6d03-4056-9984-1d50b01721a3 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.837644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1341/2657] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:06.841867 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a55781-6d03-4056-9984-1d50b01721a3 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.852599 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a55781-6d03-4056-9984-1d50b01721a3 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.857426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1948449-baa2-4f9c-b17d-7667c6c08f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.858083 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1948449-baa2-4f9c-b17d-7667c6c08f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.858361 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1948449-baa2-4f9c-b17d-7667c6c08f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.858658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1948449-baa2-4f9c-b17d-7667c6c08f77 admin admin] PATH_INFO: `/v3/projects/6ffe585843fa4042b0ecb7453eefbf14/users/0aeec5b0d57a4ac387d2d83e791e8173/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.860637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1948449-baa2-4f9c-b17d-7667c6c08f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffe585843fa4042b0ecb7453eefbf14, user_id=0aeec5b0d57a4ac387d2d83e791e8173, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.863472 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1948449-baa2-4f9c-b17d-7667c6c08f77 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.865994 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d6bfb-eb92-46ca-b988-29dfb7b0e5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.866602 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d6bfb-eb92-46ca-b988-29dfb7b0e5f9 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.866915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d6bfb-eb92-46ca-b988-29dfb7b0e5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.867189 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d6bfb-eb92-46ca-b988-29dfb7b0e5f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.867881 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d6bfb-eb92-46ca-b988-29dfb7b0e5f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.869495 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d6bfb-eb92-46ca-b988-29dfb7b0e5f9 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.874941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1342/2658] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:06 2022] PUT /identity/v3/projects/6ffe585843fa4042b0ecb7453eefbf14/users/0aeec5b0d57a4ac387d2d83e791e8173/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:58:06.879972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1317/2659] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:58:06.883563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1948449-baa2-4f9c-b17d-7667c6c08f77 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:06.887006 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357f92d-f211-44ef-a34e-dbb4f3d38a71 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.887290 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357f92d-f211-44ef-a34e-dbb4f3d38a71 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.887576 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357f92d-f211-44ef-a34e-dbb4f3d38a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.914582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-a357f92d-f211-44ef-a34e-dbb4f3d38a71 None None] MFA Rules not processed for user `0aeec5b0d57a4ac387d2d83e791e8173`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:06.915869 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a357f92d-f211-44ef-a34e-dbb4f3d38a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:06.937053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:06.953530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1948449-baa2-4f9c-b17d-7667c6c08f77 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:06.958450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215755a3-8082-4995-811f-6ec6966a0291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:06.959389 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215755a3-8082-4995-811f-6ec6966a0291 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.959784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215755a3-8082-4995-811f-6ec6966a0291 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.960055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215755a3-8082-4995-811f-6ec6966a0291 admin admin] PATH_INFO: `/v3/projects/2054ad50e510497f9e10af39051d0018/users/35249cfbeb8f4945b7d7bb43bf17575e/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:06.962060 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215755a3-8082-4995-811f-6ec6966a0291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2054ad50e510497f9e10af39051d0018, user_id=35249cfbeb8f4945b7d7bb43bf17575e, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:06.964811 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215755a3-8082-4995-811f-6ec6966a0291 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:06.979386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1343/2660] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:06 2022] PUT /identity/v3/projects/2054ad50e510497f9e10af39051d0018/users/35249cfbeb8f4945b7d7bb43bf17575e/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:06.995661 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1318/2661] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:58:06.996153 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731937bb-c18e-4652-b0c1-28863e6e049f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:06.996582 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731937bb-c18e-4652-b0c1-28863e6e049f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:06.996997 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731937bb-c18e-4652-b0c1-28863e6e049f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:07.006694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357f92d-f211-44ef-a34e-dbb4f3d38a71 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:07.008413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a357f92d-f211-44ef-a34e-dbb4f3d38a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:07.041462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-731937bb-c18e-4652-b0c1-28863e6e049f None None] MFA Rules not processed for user `35249cfbeb8f4945b7d7bb43bf17575e`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:07.042750 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-731937bb-c18e-4652-b0c1-28863e6e049f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:07.095375 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357f92d-f211-44ef-a34e-dbb4f3d38a71 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:07.116123 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1344/2662] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:06 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:07.126905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731937bb-c18e-4652-b0c1-28863e6e049f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:07.128243 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-731937bb-c18e-4652-b0c1-28863e6e049f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:07.134433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:07.149040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec602615-c7c1-4276-91f6-1d701d98ec17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:07.149659 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec602615-c7c1-4276-91f6-1d701d98ec17 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:07.149936 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec602615-c7c1-4276-91f6-1d701d98ec17 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:07.150205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec602615-c7c1-4276-91f6-1d701d98ec17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:07.188603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731937bb-c18e-4652-b0c1-28863e6e049f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:07.192215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8a458-f62a-494c-aef9-484bdf04d9a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:07.192825 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8a458-f62a-494c-aef9-484bdf04d9a4 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:07.193625 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8a458-f62a-494c-aef9-484bdf04d9a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:07.193904 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8a458-f62a-494c-aef9-484bdf04d9a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:07.202882 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8a458-f62a-494c-aef9-484bdf04d9a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:07.204306 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8a458-f62a-494c-aef9-484bdf04d9a4 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:07.211926 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec602615-c7c1-4276-91f6-1d701d98ec17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:07.213637 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec602615-c7c1-4276-91f6-1d701d98ec17 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:07.215045 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1345/2663] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:07 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:58:07.224241 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1319/2664] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:58:07.294511 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8a458-f62a-494c-aef9-484bdf04d9a4 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:07.304091 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8a458-f62a-494c-aef9-484bdf04d9a4 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:07.308070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1352b767-43d4-4b65-a259-a9148d310862 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:07.308672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352b767-43d4-4b65-a259-a9148d310862 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:07.309018 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352b767-43d4-4b65-a259-a9148d310862 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:07.309312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1352b767-43d4-4b65-a259-a9148d310862 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:07.309952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec602615-c7c1-4276-91f6-1d701d98ec17 service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:07.315202 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:07.319962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec602615-c7c1-4276-91f6-1d701d98ec17 service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:07.324118 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61db938-5a64-4cac-8508-9205b238150c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:07.324752 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61db938-5a64-4cac-8508-9205b238150c service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:07.325034 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61db938-5a64-4cac-8508-9205b238150c service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:07.325303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61db938-5a64-4cac-8508-9205b238150c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:07.368675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352b767-43d4-4b65-a259-a9148d310862 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:07.370118 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1352b767-43d4-4b65-a259-a9148d310862 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:07.379988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1346/2665] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:07 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:07.384028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61db938-5a64-4cac-8508-9205b238150c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:07.385549 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61db938-5a64-4cac-8508-9205b238150c service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:07.395600 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1320/2666] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:58:10.393478 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cea54-5bfe-46f4-a02f-5d7b1def457a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:10.393996 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cea54-5bfe-46f4-a02f-5d7b1def457a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:10.394192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cea54-5bfe-46f4-a02f-5d7b1def457a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:10.426912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-1d6cea54-5bfe-46f4-a02f-5d7b1def457a None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:10.428113 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1d6cea54-5bfe-46f4-a02f-5d7b1def457a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:10.489007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1347/2667] 10.222.0.86 () {58 vars in 967 bytes} [Fri Aug 5 19:58:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:10.537377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61db938-5a64-4cac-8508-9205b238150c service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:10.540083 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a61db938-5a64-4cac-8508-9205b238150c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:10.614881 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61db938-5a64-4cac-8508-9205b238150c service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:10.620573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3615a0f-7095-4f38-83b0-682c4558893d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:10.621184 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3615a0f-7095-4f38-83b0-682c4558893d service swift] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:10.621460 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3615a0f-7095-4f38-83b0-682c4558893d service swift] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:10.621733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3615a0f-7095-4f38-83b0-682c4558893d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:10.630908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3615a0f-7095-4f38-83b0-682c4558893d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:10.632828 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3615a0f-7095-4f38-83b0-682c4558893d service swift] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:10.641156 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1321/2668] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:58:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:58:11.658597 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cea54-5bfe-46f4-a02f-5d7b1def457a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:11.717812 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:11.741568 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cea54-5bfe-46f4-a02f-5d7b1def457a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:11.750412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:11.751236 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:11.751692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:11.752070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] PATH_INFO: `/v3/users/8928c5e7521845279b971403f223885e` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:11.753506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8928c5e7521845279b971403f223885e)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:11.755481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:11.811046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8928c5e7521845279b971403f223885e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:11.823892 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8928c5e7521845279b971403f223885e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:11.831792 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8928c5e7521845279b971403f223885e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:11.832658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8928c5e7521845279b971403f223885e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:11.833398 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8928c5e7521845279b971403f223885e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:11.847926 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1348/2669] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:11 2022] DELETE /identity/v3/users/8928c5e7521845279b971403f223885e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:11.853380 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21518c-4098-4bb9-942e-66e165207f35 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:11.854311 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21518c-4098-4bb9-942e-66e165207f35 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:11.854652 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21518c-4098-4bb9-942e-66e165207f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:11.875949 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-3b21518c-4098-4bb9-942e-66e165207f35 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:11.877022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3b21518c-4098-4bb9-942e-66e165207f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:11.943108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1322/2670] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:11.955917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:11.957901 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:12.037655 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64c8fb-ad79-4d1c-928f-21cf8eba2d40 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:12.045982 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d58f2-bb5e-4ee4-8193-3130e2caeba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:12.047191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d58f2-bb5e-4ee4-8193-3130e2caeba1 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:12.047191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d58f2-bb5e-4ee4-8193-3130e2caeba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:12.047576 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d58f2-bb5e-4ee4-8193-3130e2caeba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:12.061415 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d58f2-bb5e-4ee4-8193-3130e2caeba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:12.063039 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d58f2-bb5e-4ee4-8193-3130e2caeba1 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:12.075183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1349/2671] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:12.414485 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21518c-4098-4bb9-942e-66e165207f35 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:12.417518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3b21518c-4098-4bb9-942e-66e165207f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:12.448229 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21518c-4098-4bb9-942e-66e165207f35 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:12.453681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:12.454329 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:12.454604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:12.454902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] PATH_INFO: `/v3/projects/9c3575b5d053430a9167262299e25d25` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:12.456107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c3575b5d053430a9167262299e25d25)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:12.458063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:12.491391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:12.508742 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c3575b5d053430a9167262299e25d25', 'request_id': 'req-605f23bb-297c-43b5-8deb-6021d7b86630'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:12.524907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c3575b5d053430a9167262299e25d25', 'request_id': 'req-605f23bb-297c-43b5-8deb-6021d7b86630'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:12.540256 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c3575b5d053430a9167262299e25d25', 'request_id': 'req-605f23bb-297c-43b5-8deb-6021d7b86630'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:12.553112 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] The token cache is being invalidate because project 9c3575b5d053430a9167262299e25d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:12.555126 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:12.555514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:12.557620 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1323/2672] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:12 2022] DELETE /identity/v3/projects/9c3575b5d053430a9167262299e25d25 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:12.561624 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d58f2-bb5e-4ee4-8193-3130e2caeba1 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:12.573129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d58f2-bb5e-4ee4-8193-3130e2caeba1 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:12.655182 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156a37d-0599-4987-848f-df8d47815055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:12.656124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156a37d-0599-4987-848f-df8d47815055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:12.656511 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156a37d-0599-4987-848f-df8d47815055 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:12.656993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156a37d-0599-4987-848f-df8d47815055 admin admin] PATH_INFO: `/v3/users/35249cfbeb8f4945b7d7bb43bf17575e` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:12.658538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156a37d-0599-4987-848f-df8d47815055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35249cfbeb8f4945b7d7bb43bf17575e)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:12.660967 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156a37d-0599-4987-848f-df8d47815055 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:12.668827 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:12.740596 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9156a37d-0599-4987-848f-df8d47815055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35249cfbeb8f4945b7d7bb43bf17575e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:12.760137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9156a37d-0599-4987-848f-df8d47815055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35249cfbeb8f4945b7d7bb43bf17575e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:12.775234 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9156a37d-0599-4987-848f-df8d47815055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35249cfbeb8f4945b7d7bb43bf17575e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:12.776350 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9156a37d-0599-4987-848f-df8d47815055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35249cfbeb8f4945b7d7bb43bf17575e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:12.777289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9156a37d-0599-4987-848f-df8d47815055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35249cfbeb8f4945b7d7bb43bf17575e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:12.806773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1350/2673] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:12 2022] DELETE /identity/v3/users/35249cfbeb8f4945b7d7bb43bf17575e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:13.318213 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:13.319743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:13.344696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f23bb-297c-43b5-8deb-6021d7b86630 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:13.411489 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:13.412247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:13.412623 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:13.412960 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b admin admin] PATH_INFO: `/v3/projects/2054ad50e510497f9e10af39051d0018` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:13.415175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2054ad50e510497f9e10af39051d0018)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:13.417208 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:13.468276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2054ad50e510497f9e10af39051d0018', 'request_id': 'req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:13.475143 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2054ad50e510497f9e10af39051d0018', 'request_id': 'req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:13.489564 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2054ad50e510497f9e10af39051d0018', 'request_id': 'req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:13.502148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b admin admin] The token cache is being invalidate because project 2054ad50e510497f9e10af39051d0018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:13.502454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:13.503264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-2d01aa95-1e36-4c76-8cc6-f7b91f758e3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:13.505080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1324/2674] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:13 2022] DELETE /identity/v3/projects/2054ad50e510497f9e10af39051d0018 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:58:13.509939 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292838c-8b6e-40c0-b0f8-e47dfcc726af None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:13.510229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292838c-8b6e-40c0-b0f8-e47dfcc726af None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:13.510535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292838c-8b6e-40c0-b0f8-e47dfcc726af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:13.532494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-4292838c-8b6e-40c0-b0f8-e47dfcc726af None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:13.533713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4292838c-8b6e-40c0-b0f8-e47dfcc726af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:13.597137 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1351/2675] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:13.609459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9c0c6-329c-4410-8620-76c3af83c834 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:13.609459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9c0c6-329c-4410-8620-76c3af83c834 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:13.609459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9c0c6-329c-4410-8620-76c3af83c834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:13.633774 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-16c9c0c6-329c-4410-8620-76c3af83c834 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:13.635608 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-16c9c0c6-329c-4410-8620-76c3af83c834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:13.647303 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1325/2676] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:13.652638 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292838c-8b6e-40c0-b0f8-e47dfcc726af None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:13.662953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292838c-8b6e-40c0-b0f8-e47dfcc726af None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:13.669914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9161e-5ec1-4c3a-8fa1-a3272564091c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:13.670702 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9161e-5ec1-4c3a-8fa1-a3272564091c admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:13.671117 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9161e-5ec1-4c3a-8fa1-a3272564091c admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:13.671381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9161e-5ec1-4c3a-8fa1-a3272564091c admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:13.672283 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9161e-5ec1-4c3a-8fa1-a3272564091c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:58:13.672611 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9161e-5ec1-4c3a-8fa1-a3272564091c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:13.674206 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9161e-5ec1-4c3a-8fa1-a3272564091c admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:13.687810 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1352/2677] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:58:13.691998 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9c0c6-329c-4410-8620-76c3af83c834 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:13.702518 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9c0c6-329c-4410-8620-76c3af83c834 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:13.707476 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fd294-fcef-4f56-ae7d-fc6b981810e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:13.708093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fd294-fcef-4f56-ae7d-fc6b981810e4 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:13.708435 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fd294-fcef-4f56-ae7d-fc6b981810e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:13.708636 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fd294-fcef-4f56-ae7d-fc6b981810e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:13.709282 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546fd294-fcef-4f56-ae7d-fc6b981810e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:13.712280 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546fd294-fcef-4f56-ae7d-fc6b981810e4 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:13.750386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1326/2678] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:13 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:13.751531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9161e-5ec1-4c3a-8fa1-a3272564091c admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:13.762204 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9161e-5ec1-4c3a-8fa1-a3272564091c admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:13.821754 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8167b-4edc-45f6-a8cc-cd183981561a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:13.822508 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8167b-4edc-45f6-a8cc-cd183981561a admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:13.822819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8167b-4edc-45f6-a8cc-cd183981561a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:13.823089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8167b-4edc-45f6-a8cc-cd183981561a admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:13.823742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8167b-4edc-45f6-a8cc-cd183981561a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:13.825651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8167b-4edc-45f6-a8cc-cd183981561a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:13.847830 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:13.854726 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1353/2679] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:13 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:58:13.858740 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fd294-fcef-4f56-ae7d-fc6b981810e4 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:13.869399 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fd294-fcef-4f56-ae7d-fc6b981810e4 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:13.874454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850786a-9cda-4cec-b34a-2d741b979979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:13.875093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850786a-9cda-4cec-b34a-2d741b979979 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:13.875367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850786a-9cda-4cec-b34a-2d741b979979 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:13.875634 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850786a-9cda-4cec-b34a-2d741b979979 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:13.876391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850786a-9cda-4cec-b34a-2d741b979979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:13.877983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850786a-9cda-4cec-b34a-2d741b979979 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:13.889562 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1327/2680] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:13.893524 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8167b-4edc-45f6-a8cc-cd183981561a admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:13.904818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8167b-4edc-45f6-a8cc-cd183981561a admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:13.910064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee865724-fb96-4796-a6eb-dc3e5a86520e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:13.910803 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee865724-fb96-4796-a6eb-dc3e5a86520e admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:13.911147 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee865724-fb96-4796-a6eb-dc3e5a86520e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:13.915155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee865724-fb96-4796-a6eb-dc3e5a86520e admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:13.915155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee865724-fb96-4796-a6eb-dc3e5a86520e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:13.915155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee865724-fb96-4796-a6eb-dc3e5a86520e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:13.927038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1354/2681] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:58:13.931591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850786a-9cda-4cec-b34a-2d741b979979 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:13.942768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850786a-9cda-4cec-b34a-2d741b979979 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:13.949006 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913d953-619b-42b7-b81c-259910681d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:13.949667 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913d953-619b-42b7-b81c-259910681d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:13.949957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913d953-619b-42b7-b81c-259910681d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:13.950229 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913d953-619b-42b7-b81c-259910681d02 admin admin] PATH_INFO: `/v3/projects/b3504bc1725e441b965542a9de2216d5/users/39121e1a6ab64490b5ea8542a89dc2ab/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:13.952884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:13.970896 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913d953-619b-42b7-b81c-259910681d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3504bc1725e441b965542a9de2216d5, user_id=39121e1a6ab64490b5ea8542a89dc2ab, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:13.973897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913d953-619b-42b7-b81c-259910681d02 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:13.993220 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1328/2682] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:13 2022] PUT /identity/v3/projects/b3504bc1725e441b965542a9de2216d5/users/39121e1a6ab64490b5ea8542a89dc2ab/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:14.003689 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fba1c-7a6c-448e-8da3-6e1e53f6efa0 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:14.004045 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fba1c-7a6c-448e-8da3-6e1e53f6efa0 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:14.004421 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fba1c-7a6c-448e-8da3-6e1e53f6efa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:14.044383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-d92fba1c-7a6c-448e-8da3-6e1e53f6efa0 None None] MFA Rules not processed for user `39121e1a6ab64490b5ea8542a89dc2ab`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:14.045801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d92fba1c-7a6c-448e-8da3-6e1e53f6efa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:14.172429 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1355/2683] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:14.177223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913d953-619b-42b7-b81c-259910681d02 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:14.268880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913d953-619b-42b7-b81c-259910681d02 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:14.280875 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b86bb-9409-4ec5-b8ee-90b802a9eaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:14.281688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b86bb-9409-4ec5-b8ee-90b802a9eaa0 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:14.282073 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b86bb-9409-4ec5-b8ee-90b802a9eaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:14.282985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b86bb-9409-4ec5-b8ee-90b802a9eaa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:14.283872 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b86bb-9409-4ec5-b8ee-90b802a9eaa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:14.286322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b86bb-9409-4ec5-b8ee-90b802a9eaa0 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:14.345093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1329/2684] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:14 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:14.349766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fba1c-7a6c-448e-8da3-6e1e53f6efa0 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:14.369053 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fba1c-7a6c-448e-8da3-6e1e53f6efa0 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:14.510389 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f711d-f2e4-42ee-874b-96986591ec27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:14.511117 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f711d-f2e4-42ee-874b-96986591ec27 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:14.511401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f711d-f2e4-42ee-874b-96986591ec27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:14.511712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f711d-f2e4-42ee-874b-96986591ec27 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:14.512397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f711d-f2e4-42ee-874b-96986591ec27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:14.514717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f711d-f2e4-42ee-874b-96986591ec27 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:14.553487 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1356/2685] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:14 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:14.558185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b86bb-9409-4ec5-b8ee-90b802a9eaa0 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:14.573589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b86bb-9409-4ec5-b8ee-90b802a9eaa0 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:14.598171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1ad25-fd03-4546-84b8-6a601f2cf8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:14.600902 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1ad25-fd03-4546-84b8-6a601f2cf8af admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:14.601235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1ad25-fd03-4546-84b8-6a601f2cf8af admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:14.601515 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1ad25-fd03-4546-84b8-6a601f2cf8af admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:14.602250 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1ad25-fd03-4546-84b8-6a601f2cf8af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:14.603909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1ad25-fd03-4546-84b8-6a601f2cf8af admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:14.630422 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1330/2686] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:14.633032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f711d-f2e4-42ee-874b-96986591ec27 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:14.646901 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f711d-f2e4-42ee-874b-96986591ec27 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:14.672991 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251489a-9243-4f59-a5ca-964aa96e3f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:14.676938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251489a-9243-4f59-a5ca-964aa96e3f38 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:14.677391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251489a-9243-4f59-a5ca-964aa96e3f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:14.677669 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251489a-9243-4f59-a5ca-964aa96e3f38 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:14.678778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251489a-9243-4f59-a5ca-964aa96e3f38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:14.681294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251489a-9243-4f59-a5ca-964aa96e3f38 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:14.694390 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1357/2687] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:14.700092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1ad25-fd03-4546-84b8-6a601f2cf8af admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:14.715343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1ad25-fd03-4546-84b8-6a601f2cf8af admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:14.720718 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdbbae-8a53-40e7-8c2b-c49eafb65ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:14.721423 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdbbae-8a53-40e7-8c2b-c49eafb65ff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:14.721962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdbbae-8a53-40e7-8c2b-c49eafb65ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:14.722393 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdbbae-8a53-40e7-8c2b-c49eafb65ff5 admin admin] PATH_INFO: `/v3/projects/ac602484ba8e435fb81524f3c656391c/users/89136f4041474b7ca25a3d919d5b22ed/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:14.752082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acdbbae-8a53-40e7-8c2b-c49eafb65ff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac602484ba8e435fb81524f3c656391c, user_id=89136f4041474b7ca25a3d919d5b22ed, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:14.758368 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acdbbae-8a53-40e7-8c2b-c49eafb65ff5 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:14.805622 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1331/2688] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:14 2022] PUT /identity/v3/projects/ac602484ba8e435fb81524f3c656391c/users/89136f4041474b7ca25a3d919d5b22ed/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:14.821141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d017f-c7be-4fb9-b951-739b73a15cb1 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:14.821660 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d017f-c7be-4fb9-b951-739b73a15cb1 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:14.822121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d017f-c7be-4fb9-b951-739b73a15cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:14.885156 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-5c7d017f-c7be-4fb9-b951-739b73a15cb1 None None] MFA Rules not processed for user `89136f4041474b7ca25a3d919d5b22ed`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:14.886544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5c7d017f-c7be-4fb9-b951-739b73a15cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:14.938946 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:14.960223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1358/2689] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:14.965299 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdbbae-8a53-40e7-8c2b-c49eafb65ff5 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:15.033644 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdbbae-8a53-40e7-8c2b-c49eafb65ff5 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:15.038463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0375c-73ab-48a7-b6b8-45d8e3d3865a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:15.039684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0375c-73ab-48a7-b6b8-45d8e3d3865a admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:15.039959 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0375c-73ab-48a7-b6b8-45d8e3d3865a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:15.040403 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0375c-73ab-48a7-b6b8-45d8e3d3865a admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:15.041070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0375c-73ab-48a7-b6b8-45d8e3d3865a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:15.042992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0375c-73ab-48a7-b6b8-45d8e3d3865a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:15.071923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1332/2690] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:14 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:15.076000 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d017f-c7be-4fb9-b951-739b73a15cb1 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:15.087143 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d017f-c7be-4fb9-b951-739b73a15cb1 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:15.145470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace39133-dfe6-400d-aaba-aed7f3215ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:15.146122 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace39133-dfe6-400d-aaba-aed7f3215ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:15.146399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace39133-dfe6-400d-aaba-aed7f3215ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:15.146696 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace39133-dfe6-400d-aaba-aed7f3215ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:15.147369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace39133-dfe6-400d-aaba-aed7f3215ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:15.149737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace39133-dfe6-400d-aaba-aed7f3215ae6 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:15.170938 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1359/2691] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:15 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:58:15.174947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0375c-73ab-48a7-b6b8-45d8e3d3865a admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:15.187903 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0375c-73ab-48a7-b6b8-45d8e3d3865a admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:15.192854 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547a2c8-40af-413d-8d36-39088e7a78c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:15.193546 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547a2c8-40af-413d-8d36-39088e7a78c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:15.193898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547a2c8-40af-413d-8d36-39088e7a78c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:15.194239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547a2c8-40af-413d-8d36-39088e7a78c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:15.195651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547a2c8-40af-413d-8d36-39088e7a78c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:15.197321 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547a2c8-40af-413d-8d36-39088e7a78c8 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:15.208448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1333/2692] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:15.212708 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace39133-dfe6-400d-aaba-aed7f3215ae6 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:15.223197 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace39133-dfe6-400d-aaba-aed7f3215ae6 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:15.227875 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74a274-43fe-4ee4-bbdb-846a48312352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:15.228539 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74a274-43fe-4ee4-bbdb-846a48312352 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:15.228820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74a274-43fe-4ee4-bbdb-846a48312352 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:15.229094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74a274-43fe-4ee4-bbdb-846a48312352 admin admin] PATH_INFO: `/v3/projects/f774e21ef2c94c9a8eec1194851c63a6/users/1a0981fd9d91448dbd744253e3c22bb7/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:15.245787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74a274-43fe-4ee4-bbdb-846a48312352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f774e21ef2c94c9a8eec1194851c63a6, user_id=1a0981fd9d91448dbd744253e3c22bb7, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:15.248540 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74a274-43fe-4ee4-bbdb-846a48312352 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:15.264086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1360/2693] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:15 2022] PUT /identity/v3/projects/f774e21ef2c94c9a8eec1194851c63a6/users/1a0981fd9d91448dbd744253e3c22bb7/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:15.267646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547a2c8-40af-413d-8d36-39088e7a78c8 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:15.277823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547a2c8-40af-413d-8d36-39088e7a78c8 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:15.281605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:15.340413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75d4f3-1070-4db5-bd89-1bcecedec079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:15.340975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75d4f3-1070-4db5-bd89-1bcecedec079 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:15.341281 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75d4f3-1070-4db5-bd89-1bcecedec079 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:15.341554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75d4f3-1070-4db5-bd89-1bcecedec079 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:15.342277 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75d4f3-1070-4db5-bd89-1bcecedec079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:15.343909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75d4f3-1070-4db5-bd89-1bcecedec079 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:15.354194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1334/2694] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:15.358183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74a274-43fe-4ee4-bbdb-846a48312352 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:15.369328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74a274-43fe-4ee4-bbdb-846a48312352 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:15.374104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bc18d-7340-4cba-94d9-ae7f09f045ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:15.374856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bc18d-7340-4cba-94d9-ae7f09f045ec admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:15.375092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bc18d-7340-4cba-94d9-ae7f09f045ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:15.375503 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bc18d-7340-4cba-94d9-ae7f09f045ec admin admin] PATH_INFO: `/v3/projects/f774e21ef2c94c9a8eec1194851c63a6/users/1a0981fd9d91448dbd744253e3c22bb7/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:15.377497 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bc18d-7340-4cba-94d9-ae7f09f045ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f774e21ef2c94c9a8eec1194851c63a6, user_id=1a0981fd9d91448dbd744253e3c22bb7, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:15.380151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bc18d-7340-4cba-94d9-ae7f09f045ec admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:15.392607 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1361/2695] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:15 2022] PUT /identity/v3/projects/f774e21ef2c94c9a8eec1194851c63a6/users/1a0981fd9d91448dbd744253e3c22bb7/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:15.463725 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fdf45-d982-49cc-a930-60357a72c6ce None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:15.464015 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fdf45-d982-49cc-a930-60357a72c6ce None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:15.464274 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fdf45-d982-49cc-a930-60357a72c6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:15.496136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-cf0fdf45-d982-49cc-a930-60357a72c6ce None None] MFA Rules not processed for user `1a0981fd9d91448dbd744253e3c22bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:15.497360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-cf0fdf45-d982-49cc-a930-60357a72c6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:15.570437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1335/2696] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:58:15.582013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bc18d-7340-4cba-94d9-ae7f09f045ec admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:15.583808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-bb7bc18d-7340-4cba-94d9-ae7f09f045ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:15.645688 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bc18d-7340-4cba-94d9-ae7f09f045ec admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:15.649834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d558e-5d04-475e-9153-b54f77c473ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:15.650450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d558e-5d04-475e-9153-b54f77c473ce service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:15.650759 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d558e-5d04-475e-9153-b54f77c473ce service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:15.651030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d558e-5d04-475e-9153-b54f77c473ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:15.706400 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:15.710877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d558e-5d04-475e-9153-b54f77c473ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:15.713038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d558e-5d04-475e-9153-b54f77c473ce service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:15.723061 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1362/2697] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:17.404572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fdf45-d982-49cc-a930-60357a72c6ce None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:17.406816 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-cf0fdf45-d982-49cc-a930-60357a72c6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:17.422145 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fdf45-d982-49cc-a930-60357a72c6ce None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:17.427158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93add8d1-0810-4352-8b31-8a98edd2fc34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:17.428103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93add8d1-0810-4352-8b31-8a98edd2fc34 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:17.429533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93add8d1-0810-4352-8b31-8a98edd2fc34 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:17.430296 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93add8d1-0810-4352-8b31-8a98edd2fc34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:17.527721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93add8d1-0810-4352-8b31-8a98edd2fc34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:17.529405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93add8d1-0810-4352-8b31-8a98edd2fc34 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:17.552567 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1336/2698] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:18.462723 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d558e-5d04-475e-9153-b54f77c473ce service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:18.477391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d558e-5d04-475e-9153-b54f77c473ce service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:18.482310 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0d82e-eba1-4f3a-878a-e14039cb2128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:18.483375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0d82e-eba1-4f3a-878a-e14039cb2128 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:18.484164 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0d82e-eba1-4f3a-878a-e14039cb2128 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:18.485168 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0d82e-eba1-4f3a-878a-e14039cb2128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:18.499514 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0d82e-eba1-4f3a-878a-e14039cb2128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:18.501222 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0d82e-eba1-4f3a-878a-e14039cb2128 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:18.527679 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1363/2699] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:18.982601 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93add8d1-0810-4352-8b31-8a98edd2fc34 service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:18.984185 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-93add8d1-0810-4352-8b31-8a98edd2fc34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:19.021016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=100 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:19.077462 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93add8d1-0810-4352-8b31-8a98edd2fc34 service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:19.084463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:19.085214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:19.086077 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:19.086386 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] PATH_INFO: `/v3/users/48caa6935f9a45b994460b9cb40a4dcf` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:19.087824 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=48caa6935f9a45b994460b9cb40a4dcf)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:19.090032 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:19.163247 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48caa6935f9a45b994460b9cb40a4dcf'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.175597 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48caa6935f9a45b994460b9cb40a4dcf'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.177136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48caa6935f9a45b994460b9cb40a4dcf'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.202315 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48caa6935f9a45b994460b9cb40a4dcf'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.214035 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48caa6935f9a45b994460b9cb40a4dcf'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.225376 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1337/2700] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:18 2022] DELETE /identity/v3/users/48caa6935f9a45b994460b9cb40a4dcf => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:19.427313 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0d82e-eba1-4f3a-878a-e14039cb2128 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:19.443470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0d82e-eba1-4f3a-878a-e14039cb2128 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:19.506303 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:19.507019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:19.507305 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:19.507855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] PATH_INFO: `/v3/projects/c492b732353043b3bfe7c77daad9df4c` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:19.509245 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c492b732353043b3bfe7c77daad9df4c)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:19.511460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:19.561745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c492b732353043b3bfe7c77daad9df4c', 'request_id': 'req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.567908 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c492b732353043b3bfe7c77daad9df4c', 'request_id': 'req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.580360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c492b732353043b3bfe7c77daad9df4c', 'request_id': 'req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.591569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] The token cache is being invalidate because project c492b732353043b3bfe7c77daad9df4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:19.591855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.592502 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.594065 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1364/2701] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:19 2022] DELETE /identity/v3/projects/c492b732353043b3bfe7c77daad9df4c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:19.598438 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:19.599988 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:19.613750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02373d-6249-42e7-9dff-67cfb49070ab admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:19.671918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:19.672563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:19.672845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:19.673121 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] PATH_INFO: `/v3/users/d7d93cad817c4f42b37c897f72ba59e0` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:19.674304 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d93cad817c4f42b37c897f72ba59e0)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:19.676295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:19.732284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d93cad817c4f42b37c897f72ba59e0'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.733715 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:19.745920 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d93cad817c4f42b37c897f72ba59e0'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.746750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d93cad817c4f42b37c897f72ba59e0'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.759824 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d93cad817c4f42b37c897f72ba59e0'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.767783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d93cad817c4f42b37c897f72ba59e0'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:19.773872 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1338/2702] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:19 2022] DELETE /identity/v3/users/d7d93cad817c4f42b37c897f72ba59e0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:19.988894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:19.990574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:20.012025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a989b-f53d-4a8c-a9b1-a4c8e61b4575 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:20.083392 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:20.083951 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:20.084260 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:20.084538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] PATH_INFO: `/v3/projects/f497ae2e1f2e43c7adb89a248b86d3c6` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:20.086332 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f497ae2e1f2e43c7adb89a248b86d3c6)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:20.088426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:20.100205 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:20.136748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f497ae2e1f2e43c7adb89a248b86d3c6', 'request_id': 'req-eed30bc9-0263-4b1c-ab3d-f906e227eef9'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:20.143433 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f497ae2e1f2e43c7adb89a248b86d3c6', 'request_id': 'req-eed30bc9-0263-4b1c-ab3d-f906e227eef9'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:20.152509 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f497ae2e1f2e43c7adb89a248b86d3c6', 'request_id': 'req-eed30bc9-0263-4b1c-ab3d-f906e227eef9'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:20.164734 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] The token cache is being invalidate because project f497ae2e1f2e43c7adb89a248b86d3c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:20.165046 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:20.166079 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:20.167784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1365/2703] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:19 2022] DELETE /identity/v3/projects/f497ae2e1f2e43c7adb89a248b86d3c6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:22.047105 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:22.048523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:22.112617 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052a0b4-8156-4a38-889d-a6d15b128698 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:22.162405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa507811-614c-4221-926c-81f5fc3c77c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:22.162976 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa507811-614c-4221-926c-81f5fc3c77c5 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:22.163254 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa507811-614c-4221-926c-81f5fc3c77c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:22.163523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa507811-614c-4221-926c-81f5fc3c77c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:22.165825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-aa507811-614c-4221-926c-81f5fc3c77c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:22.236400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa507811-614c-4221-926c-81f5fc3c77c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:22.238195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa507811-614c-4221-926c-81f5fc3c77c5 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:22.250300 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1339/2704] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:22.471763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:22.473212 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:22.545126 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed30bc9-0263-4b1c-ab3d-f906e227eef9 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:22.550432 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:22.551111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:22.551406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:22.551653 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] PATH_INFO: `/v3/users/39121e1a6ab64490b5ea8542a89dc2ab` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:22.552833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39121e1a6ab64490b5ea8542a89dc2ab)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:22.554836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:22.605121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39121e1a6ab64490b5ea8542a89dc2ab'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:22.618564 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39121e1a6ab64490b5ea8542a89dc2ab'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:22.626446 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39121e1a6ab64490b5ea8542a89dc2ab'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:22.627103 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39121e1a6ab64490b5ea8542a89dc2ab'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:22.627946 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39121e1a6ab64490b5ea8542a89dc2ab'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:22.640869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1366/2705] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:22 2022] DELETE /identity/v3/users/39121e1a6ab64490b5ea8542a89dc2ab => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:22.645317 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1507ae-eaf9-46f8-99f7-556f976b1321 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:22.645663 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1507ae-eaf9-46f8-99f7-556f976b1321 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:22.646180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1507ae-eaf9-46f8-99f7-556f976b1321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:22.666289 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-af1507ae-eaf9-46f8-99f7-556f976b1321 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:22.667573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-af1507ae-eaf9-46f8-99f7-556f976b1321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:22.694149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:22.733362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1340/2706] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:58:22.743211 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:22.744652 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:22.774316 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:22.816682 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8f4c0-be6d-46be-9839-7a5a6bea5866 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:22.820535 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5d696-8f17-4ada-b5a1-60600908ef27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:22.821202 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5d696-8f17-4ada-b5a1-60600908ef27 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:22.821482 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5d696-8f17-4ada-b5a1-60600908ef27 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:22.821751 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5d696-8f17-4ada-b5a1-60600908ef27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:22.832593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5d696-8f17-4ada-b5a1-60600908ef27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:22.834082 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f5d696-8f17-4ada-b5a1-60600908ef27 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:22.844929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1367/2707] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:23.161948 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1507ae-eaf9-46f8-99f7-556f976b1321 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:23.164556 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-af1507ae-eaf9-46f8-99f7-556f976b1321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:23.185166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1507ae-eaf9-46f8-99f7-556f976b1321 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:23.190195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:23.190896 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:23.191520 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:23.191958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] PATH_INFO: `/v3/projects/b3504bc1725e441b965542a9de2216d5` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:23.199859 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3504bc1725e441b965542a9de2216d5)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:23.202336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:23.259765 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3504bc1725e441b965542a9de2216d5', 'request_id': 'req-1fcfbfa2-f690-46ae-8b87-682eebcd3839'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.268042 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3504bc1725e441b965542a9de2216d5', 'request_id': 'req-1fcfbfa2-f690-46ae-8b87-682eebcd3839'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.278584 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3504bc1725e441b965542a9de2216d5', 'request_id': 'req-1fcfbfa2-f690-46ae-8b87-682eebcd3839'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.292747 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] The token cache is being invalidate because project b3504bc1725e441b965542a9de2216d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:23.293510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.294931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.296932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1341/2708] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:23 2022] DELETE /identity/v3/projects/b3504bc1725e441b965542a9de2216d5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:23.300915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5d696-8f17-4ada-b5a1-60600908ef27 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:23.311764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5d696-8f17-4ada-b5a1-60600908ef27 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:23.378653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:23.380160 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:23.389367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8f819-a278-4427-b044-ea3e6dbc8324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:23.390017 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8f819-a278-4427-b044-ea3e6dbc8324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:23.390318 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8f819-a278-4427-b044-ea3e6dbc8324 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:23.390618 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8f819-a278-4427-b044-ea3e6dbc8324 admin admin] PATH_INFO: `/v3/users/89136f4041474b7ca25a3d919d5b22ed` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:23.391899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8f819-a278-4427-b044-ea3e6dbc8324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89136f4041474b7ca25a3d919d5b22ed)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:23.393891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8f819-a278-4427-b044-ea3e6dbc8324 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:23.447043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-34f8f819-a278-4427-b044-ea3e6dbc8324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89136f4041474b7ca25a3d919d5b22ed'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.460209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-34f8f819-a278-4427-b044-ea3e6dbc8324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89136f4041474b7ca25a3d919d5b22ed'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.474585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-34f8f819-a278-4427-b044-ea3e6dbc8324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89136f4041474b7ca25a3d919d5b22ed'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.479181 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-34f8f819-a278-4427-b044-ea3e6dbc8324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89136f4041474b7ca25a3d919d5b22ed'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.480127 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-34f8f819-a278-4427-b044-ea3e6dbc8324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89136f4041474b7ca25a3d919d5b22ed'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.491093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfbfa2-f690-46ae-8b87-682eebcd3839 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:23.497578 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1368/2709] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:23 2022] DELETE /identity/v3/users/89136f4041474b7ca25a3d919d5b22ed => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:23.498592 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:23.499448 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:23.499870 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:23.500330 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] PATH_INFO: `/v3/users/b700b468142b4ae88b17efa0112747f0` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:23.502529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b700b468142b4ae88b17efa0112747f0)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:23.509346 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:23.573557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b700b468142b4ae88b17efa0112747f0'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.583957 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b700b468142b4ae88b17efa0112747f0'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.584849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b700b468142b4ae88b17efa0112747f0'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.587897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:23.604624 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b700b468142b4ae88b17efa0112747f0'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.615141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b700b468142b4ae88b17efa0112747f0'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.625609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1342/2710] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:23 2022] DELETE /identity/v3/users/b700b468142b4ae88b17efa0112747f0 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:23.627412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc5e37-23ef-45d3-99ea-6df248bf62ed None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:23.627698 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc5e37-23ef-45d3-99ea-6df248bf62ed None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:23.627983 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc5e37-23ef-45d3-99ea-6df248bf62ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:23.647356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-7bdc5e37-23ef-45d3-99ea-6df248bf62ed None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:23.649332 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7bdc5e37-23ef-45d3-99ea-6df248bf62ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:23.693489 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:23.722788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1369/2711] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:23.734082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:23.735605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:23.809473 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec4aa0-f8e8-4ad5-93bf-a53c4ea076f0 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:23.813800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f8958-66cf-4f15-9fb1-a36c8e687376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:23.814428 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f8958-66cf-4f15-9fb1-a36c8e687376 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:23.814738 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f8958-66cf-4f15-9fb1-a36c8e687376 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:23.815008 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f8958-66cf-4f15-9fb1-a36c8e687376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:23.826107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f8958-66cf-4f15-9fb1-a36c8e687376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:23.827622 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f8958-66cf-4f15-9fb1-a36c8e687376 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:23.839455 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc5e37-23ef-45d3-99ea-6df248bf62ed None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:23.839899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1343/2712] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:23.841393 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7bdc5e37-23ef-45d3-99ea-6df248bf62ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:23.854538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc5e37-23ef-45d3-99ea-6df248bf62ed None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:23.860130 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:23.860997 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:23.861434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:23.861844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] PATH_INFO: `/v3/projects/ac602484ba8e435fb81524f3c656391c` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:23.863254 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac602484ba8e435fb81524f3c656391c)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:23.865153 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:23.918888 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac602484ba8e435fb81524f3c656391c', 'request_id': 'req-d0fb1989-be34-4b4f-b301-4cd054bb847e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.925649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac602484ba8e435fb81524f3c656391c', 'request_id': 'req-d0fb1989-be34-4b4f-b301-4cd054bb847e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.935774 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac602484ba8e435fb81524f3c656391c', 'request_id': 'req-d0fb1989-be34-4b4f-b301-4cd054bb847e'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.947919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] The token cache is being invalidate because project ac602484ba8e435fb81524f3c656391c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:23.948154 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.948782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:23.950323 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1370/2713] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:23 2022] DELETE /identity/v3/projects/ac602484ba8e435fb81524f3c656391c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:23.954503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f8958-66cf-4f15-9fb1-a36c8e687376 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:23.966235 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f8958-66cf-4f15-9fb1-a36c8e687376 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:24.036439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:24.037413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:24.037692 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:24.037990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] PATH_INFO: `/v3/users/1a0981fd9d91448dbd744253e3c22bb7` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:24.039985 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a0981fd9d91448dbd744253e3c22bb7)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:24.042400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:24.096302 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a0981fd9d91448dbd744253e3c22bb7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.105760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a0981fd9d91448dbd744253e3c22bb7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.107096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a0981fd9d91448dbd744253e3c22bb7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.123906 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a0981fd9d91448dbd744253e3c22bb7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.135221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:24.141684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a0981fd9d91448dbd744253e3c22bb7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.155620 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1344/2714] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:23 2022] DELETE /identity/v3/users/1a0981fd9d91448dbd744253e3c22bb7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:24.521868 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:24.525470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:24.544942 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb1989-be34-4b4f-b301-4cd054bb847e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:24.645776 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:24.646790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:24.647241 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:24.647664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] PATH_INFO: `/v3/projects/87418eff0be4435696b0b1b8c64c32ee` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:24.649155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87418eff0be4435696b0b1b8c64c32ee)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:24.651362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:24.717780 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87418eff0be4435696b0b1b8c64c32ee', 'request_id': 'req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.727824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87418eff0be4435696b0b1b8c64c32ee', 'request_id': 'req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.733455 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:24.735190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:24.743497 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87418eff0be4435696b0b1b8c64c32ee', 'request_id': 'req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.750674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:24.750843 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa7aa70-fce5-45aa-a4d1-981f5ba8eb9f admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:24.776019 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] The token cache is being invalidate because project 87418eff0be4435696b0b1b8c64c32ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:24.776316 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.777207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.783453 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1371/2715] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:24 2022] DELETE /identity/v3/projects/87418eff0be4435696b0b1b8c64c32ee => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:24.784494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:24.786402 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:24.858558 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb429e-7815-401c-bfb6-fcef35c10dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:24.859411 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb429e-7815-401c-bfb6-fcef35c10dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:24.860111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb429e-7815-401c-bfb6-fcef35c10dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:24.860255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb429e-7815-401c-bfb6-fcef35c10dc9 admin admin] PATH_INFO: `/v3/projects/f774e21ef2c94c9a8eec1194851c63a6` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:24.862055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb429e-7815-401c-bfb6-fcef35c10dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f774e21ef2c94c9a8eec1194851c63a6)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:24.864205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb429e-7815-401c-bfb6-fcef35c10dc9 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:24.880729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6facb95-dbc7-42fb-a7a4-fc77fdb00a68 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:24.888729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e0760-ab4a-44e2-8e43-92ef76f178bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:24.889463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e0760-ab4a-44e2-8e43-92ef76f178bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:24.889935 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e0760-ab4a-44e2-8e43-92ef76f178bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:24.890280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e0760-ab4a-44e2-8e43-92ef76f178bb admin admin] PATH_INFO: `/v3/users/0aeec5b0d57a4ac387d2d83e791e8173` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:24.891612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e0760-ab4a-44e2-8e43-92ef76f178bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aeec5b0d57a4ac387d2d83e791e8173)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:24.894195 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e0760-ab4a-44e2-8e43-92ef76f178bb admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:24.943313 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-21eb429e-7815-401c-bfb6-fcef35c10dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f774e21ef2c94c9a8eec1194851c63a6', 'request_id': 'req-21eb429e-7815-401c-bfb6-fcef35c10dc9'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.951151 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-21eb429e-7815-401c-bfb6-fcef35c10dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f774e21ef2c94c9a8eec1194851c63a6', 'request_id': 'req-21eb429e-7815-401c-bfb6-fcef35c10dc9'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.966125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-21eb429e-7815-401c-bfb6-fcef35c10dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f774e21ef2c94c9a8eec1194851c63a6', 'request_id': 'req-21eb429e-7815-401c-bfb6-fcef35c10dc9'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.983347 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-21eb429e-7815-401c-bfb6-fcef35c10dc9 admin admin] The token cache is being invalidate because project f774e21ef2c94c9a8eec1194851c63a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:24.983672 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-21eb429e-7815-401c-bfb6-fcef35c10dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.984424 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-21eb429e-7815-401c-bfb6-fcef35c10dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.985204 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9d1e0760-ab4a-44e2-8e43-92ef76f178bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aeec5b0d57a4ac387d2d83e791e8173'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:24.989488 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1345/2716] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:24 2022] DELETE /identity/v3/projects/f774e21ef2c94c9a8eec1194851c63a6 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:24.990850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd002b5-c139-455b-a6b3-43e32fffd29a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:24.991138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd002b5-c139-455b-a6b3-43e32fffd29a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:24.991416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd002b5-c139-455b-a6b3-43e32fffd29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.002416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9d1e0760-ab4a-44e2-8e43-92ef76f178bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aeec5b0d57a4ac387d2d83e791e8173'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:25.012736 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9d1e0760-ab4a-44e2-8e43-92ef76f178bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aeec5b0d57a4ac387d2d83e791e8173'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:25.013279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-fbd002b5-c139-455b-a6b3-43e32fffd29a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:25.014090 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9d1e0760-ab4a-44e2-8e43-92ef76f178bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aeec5b0d57a4ac387d2d83e791e8173'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:25.014605 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-fbd002b5-c139-455b-a6b3-43e32fffd29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:25.015851 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9d1e0760-ab4a-44e2-8e43-92ef76f178bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aeec5b0d57a4ac387d2d83e791e8173'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:25.037533 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1372/2717] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:24 2022] DELETE /identity/v3/users/0aeec5b0d57a4ac387d2d83e791e8173 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:25.098734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1346/2718] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:25.107383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0110b-93f2-4b89-b8a3-c00e65db8980 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.107632 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0110b-93f2-4b89-b8a3-c00e65db8980 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.108034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0110b-93f2-4b89-b8a3-c00e65db8980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.133731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-d5d0110b-93f2-4b89-b8a3-c00e65db8980 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:25.134967 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d5d0110b-93f2-4b89-b8a3-c00e65db8980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:25.216490 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1373/2719] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:25.221124 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd002b5-c139-455b-a6b3-43e32fffd29a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.246222 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd002b5-c139-455b-a6b3-43e32fffd29a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:25.252750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c111774-5e6c-4e98-b4ca-d99c29c42f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:25.253589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111774-5e6c-4e98-b4ca-d99c29c42f32 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.254018 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111774-5e6c-4e98-b4ca-d99c29c42f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.254447 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111774-5e6c-4e98-b4ca-d99c29c42f32 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.255049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c111774-5e6c-4e98-b4ca-d99c29c42f32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:58:25.255353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c111774-5e6c-4e98-b4ca-d99c29c42f32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:25.257251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c111774-5e6c-4e98-b4ca-d99c29c42f32 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:25.271415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1347/2720] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:58:25.276333 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0110b-93f2-4b89-b8a3-c00e65db8980 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.285617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0110b-93f2-4b89-b8a3-c00e65db8980 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:25.292838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f4f2f-1139-4a36-b464-c762b4cd36ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:25.292838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f4f2f-1139-4a36-b464-c762b4cd36ca admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.292838 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f4f2f-1139-4a36-b464-c762b4cd36ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.293614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f4f2f-1139-4a36-b464-c762b4cd36ca admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.294034 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f4f2f-1139-4a36-b464-c762b4cd36ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:25.296004 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f4f2f-1139-4a36-b464-c762b4cd36ca admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:25.312584 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:25.329182 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1374/2721] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:25 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:25.334164 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c111774-5e6c-4e98-b4ca-d99c29c42f32 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.347352 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c111774-5e6c-4e98-b4ca-d99c29c42f32 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:25.361862 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:25.376335 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f4f2f-1139-4a36-b464-c762b4cd36ca admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.378155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-072f4f2f-1139-4a36-b464-c762b4cd36ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:25.463255 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f4f2f-1139-4a36-b464-c762b4cd36ca admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:25.470912 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b25de2-9b5d-4e46-a94e-76d084557020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:25.471664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b25de2-9b5d-4e46-a94e-76d084557020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.471989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b25de2-9b5d-4e46-a94e-76d084557020 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.472267 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b25de2-9b5d-4e46-a94e-76d084557020 admin admin] PATH_INFO: `/v3/projects/6ffe585843fa4042b0ecb7453eefbf14` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.473659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b25de2-9b5d-4e46-a94e-76d084557020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ffe585843fa4042b0ecb7453eefbf14)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:25.479469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b25de2-9b5d-4e46-a94e-76d084557020 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:25.535659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-64b25de2-9b5d-4e46-a94e-76d084557020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ffe585843fa4042b0ecb7453eefbf14', 'request_id': 'req-64b25de2-9b5d-4e46-a94e-76d084557020'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:25.542988 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-64b25de2-9b5d-4e46-a94e-76d084557020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ffe585843fa4042b0ecb7453eefbf14', 'request_id': 'req-64b25de2-9b5d-4e46-a94e-76d084557020'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:25.552004 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-64b25de2-9b5d-4e46-a94e-76d084557020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ffe585843fa4042b0ecb7453eefbf14', 'request_id': 'req-64b25de2-9b5d-4e46-a94e-76d084557020'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:25.559795 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2b832-44e6-434f-96ee-21b70a8b281d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:25.560559 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2b832-44e6-434f-96ee-21b70a8b281d admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.560884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2b832-44e6-434f-96ee-21b70a8b281d admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.561153 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2b832-44e6-434f-96ee-21b70a8b281d admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.562141 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2b832-44e6-434f-96ee-21b70a8b281d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:25.563820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-64b25de2-9b5d-4e46-a94e-76d084557020 admin admin] The token cache is being invalidate because project 6ffe585843fa4042b0ecb7453eefbf14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:25.564111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-64b25de2-9b5d-4e46-a94e-76d084557020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:25.564857 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2b832-44e6-434f-96ee-21b70a8b281d admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:25.564970 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-64b25de2-9b5d-4e46-a94e-76d084557020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:25.566538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1375/2722] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:25 2022] DELETE /identity/v3/projects/6ffe585843fa4042b0ecb7453eefbf14 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:25.572136 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9770b6f-8d95-4c06-9089-4cf4bfcb3f83 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.572420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9770b6f-8d95-4c06-9089-4cf4bfcb3f83 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.572695 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9770b6f-8d95-4c06-9089-4cf4bfcb3f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.593461 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1348/2723] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:25 2022] POST /identity/v3/users => generated 286 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:25.594564 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2b832-44e6-434f-96ee-21b70a8b281d admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.596343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-5ab2b832-44e6-434f-96ee-21b70a8b281d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:25.601953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-a9770b6f-8d95-4c06-9089-4cf4bfcb3f83 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:25.603259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a9770b6f-8d95-4c06-9089-4cf4bfcb3f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:25.608579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2b832-44e6-434f-96ee-21b70a8b281d admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:25.665429 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1376/2724] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:58:25.667763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35752601-2935-4ef1-991e-7ddac6d2a621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:25.668615 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35752601-2935-4ef1-991e-7ddac6d2a621 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.669050 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35752601-2935-4ef1-991e-7ddac6d2a621 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.669475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35752601-2935-4ef1-991e-7ddac6d2a621 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.670489 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35752601-2935-4ef1-991e-7ddac6d2a621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:25.672624 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35752601-2935-4ef1-991e-7ddac6d2a621 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:25.673150 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139d526-34f2-4cb8-affe-c2bdae4f6f4f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.673439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139d526-34f2-4cb8-affe-c2bdae4f6f4f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.673712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139d526-34f2-4cb8-affe-c2bdae4f6f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.685973 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1349/2725] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:58:25.690147 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35752601-2935-4ef1-991e-7ddac6d2a621 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.694787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-6139d526-34f2-4cb8-affe-c2bdae4f6f4f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:25.696647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6139d526-34f2-4cb8-affe-c2bdae4f6f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:25.703799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35752601-2935-4ef1-991e-7ddac6d2a621 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:25.704891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1377/2726] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:58:25.708956 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc9820-3d1f-4091-9ec9-286a3aadbac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:25.709274 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139d526-34f2-4cb8-affe-c2bdae4f6f4f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.709865 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc9820-3d1f-4091-9ec9-286a3aadbac1 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.710156 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc9820-3d1f-4091-9ec9-286a3aadbac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.710480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc9820-3d1f-4091-9ec9-286a3aadbac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.711783 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc9820-3d1f-4091-9ec9-286a3aadbac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:25.713584 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc9820-3d1f-4091-9ec9-286a3aadbac1 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:25.720483 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139d526-34f2-4cb8-affe-c2bdae4f6f4f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:25.724832 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1350/2727] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:58:25.726322 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3668d29-afd8-4046-b766-66255f6e138a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:25.727531 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3668d29-afd8-4046-b766-66255f6e138a admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.727797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3668d29-afd8-4046-b766-66255f6e138a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.728057 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3668d29-afd8-4046-b766-66255f6e138a admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.728755 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3668d29-afd8-4046-b766-66255f6e138a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:58:25.729221 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3668d29-afd8-4046-b766-66255f6e138a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:25.730040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc9820-3d1f-4091-9ec9-286a3aadbac1 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.730695 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3668d29-afd8-4046-b766-66255f6e138a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:25.739787 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc9820-3d1f-4091-9ec9-286a3aadbac1 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:25.742664 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1378/2728] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:58:25.747027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3668d29-afd8-4046-b766-66255f6e138a admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.749011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef15661-046c-457f-9288-74cadc6dabe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:25.749668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef15661-046c-457f-9288-74cadc6dabe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.749953 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef15661-046c-457f-9288-74cadc6dabe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.750228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef15661-046c-457f-9288-74cadc6dabe1 admin admin] PATH_INFO: `/v3/projects/1a0f6dc247f547a79f8908d072bc49e3/users/2df1b8bc5012478b9ac5b4c2341a8d12/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.753821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:25.758073 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3668d29-afd8-4046-b766-66255f6e138a admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:25.765383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70579bed-f59a-4ecd-b1c5-e42f84ca699e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:25.766244 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70579bed-f59a-4ecd-b1c5-e42f84ca699e admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.766391 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef15661-046c-457f-9288-74cadc6dabe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0f6dc247f547a79f8908d072bc49e3, user_id=2df1b8bc5012478b9ac5b4c2341a8d12, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:25.766658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70579bed-f59a-4ecd-b1c5-e42f84ca699e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.767170 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70579bed-f59a-4ecd-b1c5-e42f84ca699e admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.767833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70579bed-f59a-4ecd-b1c5-e42f84ca699e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:25.769190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef15661-046c-457f-9288-74cadc6dabe1 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:25.769785 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70579bed-f59a-4ecd-b1c5-e42f84ca699e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:25.783514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1351/2729] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:25 2022] PUT /identity/v3/projects/1a0f6dc247f547a79f8908d072bc49e3/users/2df1b8bc5012478b9ac5b4c2341a8d12/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:58:25.788465 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bdcd0-e3a3-4874-b2cc-0a0e2fe6d9df None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.788775 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bdcd0-e3a3-4874-b2cc-0a0e2fe6d9df None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.789797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bdcd0-e3a3-4874-b2cc-0a0e2fe6d9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.800435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1379/2730] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:25 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:58:25.805162 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70579bed-f59a-4ecd-b1c5-e42f84ca699e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.809472 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-145bdcd0-e3a3-4874-b2cc-0a0e2fe6d9df None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:25.810703 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-145bdcd0-e3a3-4874-b2cc-0a0e2fe6d9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:25.870463 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70579bed-f59a-4ecd-b1c5-e42f84ca699e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:25.871700 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1352/2731] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:58:25.875196 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253f11b-b34e-4b15-8db0-77f5a2964cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:25.875808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253f11b-b34e-4b15-8db0-77f5a2964cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.876086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253f11b-b34e-4b15-8db0-77f5a2964cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.876359 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253f11b-b34e-4b15-8db0-77f5a2964cf8 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.877054 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253f11b-b34e-4b15-8db0-77f5a2964cf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:25.879004 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253f11b-b34e-4b15-8db0-77f5a2964cf8 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:25.882337 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bdcd0-e3a3-4874-b2cc-0a0e2fe6d9df None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.885158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-145bdcd0-e3a3-4874-b2cc-0a0e2fe6d9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:25.891315 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:25.904014 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1380/2732] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:25 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:58:25.907943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253f11b-b34e-4b15-8db0-77f5a2964cf8 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.919482 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253f11b-b34e-4b15-8db0-77f5a2964cf8 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:25.924372 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954ae91-e400-4d8d-a4b5-077d79ea912e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:25.924983 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954ae91-e400-4d8d-a4b5-077d79ea912e admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.925261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954ae91-e400-4d8d-a4b5-077d79ea912e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.925612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e954ae91-e400-4d8d-a4b5-077d79ea912e admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.926480 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e954ae91-e400-4d8d-a4b5-077d79ea912e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:25.928339 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e954ae91-e400-4d8d-a4b5-077d79ea912e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:25.942406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1381/2733] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:58:25.943484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954ae91-e400-4d8d-a4b5-077d79ea912e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.952159 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bdcd0-e3a3-4874-b2cc-0a0e2fe6d9df None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:25.954673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e954ae91-e400-4d8d-a4b5-077d79ea912e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:25.955785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46395a19-b0f9-4105-83e6-38c6140b7291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:25.956404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46395a19-b0f9-4105-83e6-38c6140b7291 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.956679 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46395a19-b0f9-4105-83e6-38c6140b7291 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.956952 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46395a19-b0f9-4105-83e6-38c6140b7291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.959741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d960f5-4c31-4330-b6ef-6d92609c0cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:25.960338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d960f5-4c31-4330-b6ef-6d92609c0cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:25.960631 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d960f5-4c31-4330-b6ef-6d92609c0cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:25.960955 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d960f5-4c31-4330-b6ef-6d92609c0cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:25.962807 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d960f5-4c31-4330-b6ef-6d92609c0cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:25.964544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d960f5-4c31-4330-b6ef-6d92609c0cb9 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:25.967407 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46395a19-b0f9-4105-83e6-38c6140b7291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:25.968788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46395a19-b0f9-4105-83e6-38c6140b7291 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:25.978413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1382/2734] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:25.983132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d960f5-4c31-4330-b6ef-6d92609c0cb9 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:25.983435 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1353/2735] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:58:25.993516 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d960f5-4c31-4330-b6ef-6d92609c0cb9 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:26.000862 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee807a3-fe01-460d-bef8-0dde4d24d5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:26.001529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee807a3-fe01-460d-bef8-0dde4d24d5c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:26.001806 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee807a3-fe01-460d-bef8-0dde4d24d5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:26.002076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee807a3-fe01-460d-bef8-0dde4d24d5c1 admin admin] PATH_INFO: `/v3/projects/372345642a564430b9b41421ffde0afa/users/8b4cfeab438c41b7914ea577649c0aae/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:26.017832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee807a3-fe01-460d-bef8-0dde4d24d5c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=372345642a564430b9b41421ffde0afa, user_id=8b4cfeab438c41b7914ea577649c0aae, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:26.020579 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee807a3-fe01-460d-bef8-0dde4d24d5c1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:26.035471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1383/2736] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:25 2022] PUT /identity/v3/projects/372345642a564430b9b41421ffde0afa/users/8b4cfeab438c41b7914ea577649c0aae/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:58:26.047822 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbb785-35c2-4ca5-b347-f5729a10bb76 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:26.048114 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbb785-35c2-4ca5-b347-f5729a10bb76 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:26.048387 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbb785-35c2-4ca5-b347-f5729a10bb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:26.080999 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-f8dbb785-35c2-4ca5-b347-f5729a10bb76 None None] MFA Rules not processed for user `8b4cfeab438c41b7914ea577649c0aae`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:26.082548 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f8dbb785-35c2-4ca5-b347-f5729a10bb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:26.156900 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1354/2737] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:26.161500 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee807a3-fe01-460d-bef8-0dde4d24d5c1 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:26.228913 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee807a3-fe01-460d-bef8-0dde4d24d5c1 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:26.233730 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6b234-43b7-402a-a17d-353fcf73ee38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:26.234355 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6b234-43b7-402a-a17d-353fcf73ee38 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:26.234672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6b234-43b7-402a-a17d-353fcf73ee38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:26.235043 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6b234-43b7-402a-a17d-353fcf73ee38 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:26.235763 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6b234-43b7-402a-a17d-353fcf73ee38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:26.237668 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6b234-43b7-402a-a17d-353fcf73ee38 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:26.266826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1384/2738] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:26 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:26.271035 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbb785-35c2-4ca5-b347-f5729a10bb76 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:26.284103 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbb785-35c2-4ca5-b347-f5729a10bb76 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:26.365180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20ae8a-b07f-43f5-8d2e-5c6827f5f634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:26.366035 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20ae8a-b07f-43f5-8d2e-5c6827f5f634 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:26.366383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20ae8a-b07f-43f5-8d2e-5c6827f5f634 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:26.366757 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20ae8a-b07f-43f5-8d2e-5c6827f5f634 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:26.367668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20ae8a-b07f-43f5-8d2e-5c6827f5f634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:26.369754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20ae8a-b07f-43f5-8d2e-5c6827f5f634 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:26.394665 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1355/2739] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:26 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:26.399182 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6b234-43b7-402a-a17d-353fcf73ee38 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:26.413332 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6b234-43b7-402a-a17d-353fcf73ee38 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:26.425378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b352-972d-4649-b56c-3e98a0c6ee13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:26.425984 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b352-972d-4649-b56c-3e98a0c6ee13 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:26.426273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b352-972d-4649-b56c-3e98a0c6ee13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:26.426553 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b352-972d-4649-b56c-3e98a0c6ee13 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:26.427293 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32b352-972d-4649-b56c-3e98a0c6ee13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:26.428989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32b352-972d-4649-b56c-3e98a0c6ee13 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:26.435623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:26.442907 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1385/2740] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:26.446776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20ae8a-b07f-43f5-8d2e-5c6827f5f634 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:26.458122 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20ae8a-b07f-43f5-8d2e-5c6827f5f634 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:26.463776 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3283c1-8d51-4fdf-b000-fc3604020292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:26.464560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3283c1-8d51-4fdf-b000-fc3604020292 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:26.464918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3283c1-8d51-4fdf-b000-fc3604020292 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:26.465243 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3283c1-8d51-4fdf-b000-fc3604020292 admin admin] PATH_INFO: `/v3/projects/4739b7d431da4b058fc048cfe74e0537/users/a7d70a286c6348d3adec201d22af811a/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:26.482779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3283c1-8d51-4fdf-b000-fc3604020292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4739b7d431da4b058fc048cfe74e0537, user_id=a7d70a286c6348d3adec201d22af811a, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:26.488365 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3283c1-8d51-4fdf-b000-fc3604020292 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:26.503387 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1356/2741] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:26 2022] PUT /identity/v3/projects/4739b7d431da4b058fc048cfe74e0537/users/a7d70a286c6348d3adec201d22af811a/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:26.507434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b352-972d-4649-b56c-3e98a0c6ee13 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:26.517619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b352-972d-4649-b56c-3e98a0c6ee13 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:26.576559 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35722ba-13e6-42b3-9ad9-e5fd2e1d59a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:26.577290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35722ba-13e6-42b3-9ad9-e5fd2e1d59a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:26.577966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35722ba-13e6-42b3-9ad9-e5fd2e1d59a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:26.577966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35722ba-13e6-42b3-9ad9-e5fd2e1d59a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:26.578635 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35722ba-13e6-42b3-9ad9-e5fd2e1d59a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:26.580414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35722ba-13e6-42b3-9ad9-e5fd2e1d59a6 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:26.591795 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1386/2742] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:58:26.596016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3283c1-8d51-4fdf-b000-fc3604020292 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:26.602983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:26.610323 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3283c1-8d51-4fdf-b000-fc3604020292 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:26.616138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf2055-b682-438d-8479-9aa21eedbbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:26.617029 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf2055-b682-438d-8479-9aa21eedbbbd admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:26.617325 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf2055-b682-438d-8479-9aa21eedbbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:26.617709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf2055-b682-438d-8479-9aa21eedbbbd admin admin] PATH_INFO: `/v3/projects/4739b7d431da4b058fc048cfe74e0537/users/a7d70a286c6348d3adec201d22af811a/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:26.620279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf2055-b682-438d-8479-9aa21eedbbbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4739b7d431da4b058fc048cfe74e0537, user_id=a7d70a286c6348d3adec201d22af811a, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:26.622991 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf2055-b682-438d-8479-9aa21eedbbbd admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:26.635111 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1357/2743] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:26 2022] PUT /identity/v3/projects/4739b7d431da4b058fc048cfe74e0537/users/a7d70a286c6348d3adec201d22af811a/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:26.645378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ea042-94b8-4597-a153-6c5e52775b53 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:26.645672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ea042-94b8-4597-a153-6c5e52775b53 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:26.645947 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ea042-94b8-4597-a153-6c5e52775b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:26.686141 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-ae8ea042-94b8-4597-a153-6c5e52775b53 None None] MFA Rules not processed for user `a7d70a286c6348d3adec201d22af811a`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:26.686353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73e7d8-3c92-45ff-b067-a4cbc0ab11c5 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:26.686755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73e7d8-3c92-45ff-b067-a4cbc0ab11c5 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:26.687127 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73e7d8-3c92-45ff-b067-a4cbc0ab11c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:26.687362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ae8ea042-94b8-4597-a153-6c5e52775b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:26.721620 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-fa73e7d8-3c92-45ff-b067-a4cbc0ab11c5 None None] MFA Rules not processed for user `2df1b8bc5012478b9ac5b4c2341a8d12`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:26.723134 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-fa73e7d8-3c92-45ff-b067-a4cbc0ab11c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:26.770191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1387/2744] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:26.781859 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ea042-94b8-4597-a153-6c5e52775b53 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:26.783287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ae8ea042-94b8-4597-a153-6c5e52775b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:26.806606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1358/2745] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:58:26.815469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73e7d8-3c92-45ff-b067-a4cbc0ab11c5 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:26.817259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-fa73e7d8-3c92-45ff-b067-a4cbc0ab11c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:26.859140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ea042-94b8-4597-a153-6c5e52775b53 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:26.860474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0477b8-2083-421b-a232-63b3ffaff4ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:26.861317 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0477b8-2083-421b-a232-63b3ffaff4ed service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:26.861699 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0477b8-2083-421b-a232-63b3ffaff4ed service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:26.861941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0477b8-2083-421b-a232-63b3ffaff4ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:26.872937 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0477b8-2083-421b-a232-63b3ffaff4ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:26.874649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0477b8-2083-421b-a232-63b3ffaff4ed service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:26.886247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1388/2746] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:58:26.887554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73e7d8-3c92-45ff-b067-a4cbc0ab11c5 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:26.891283 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b201fa-9eea-40c4-b13e-936ea6aeb179 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:26.891915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b201fa-9eea-40c4-b13e-936ea6aeb179 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:26.892198 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b201fa-9eea-40c4-b13e-936ea6aeb179 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:26.892472 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b201fa-9eea-40c4-b13e-936ea6aeb179 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:26.901729 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b201fa-9eea-40c4-b13e-936ea6aeb179 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:26.903336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b201fa-9eea-40c4-b13e-936ea6aeb179 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:26.916229 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1359/2747] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 19:58:26.970870 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0477b8-2083-421b-a232-63b3ffaff4ed service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:26.980845 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0477b8-2083-421b-a232-63b3ffaff4ed service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:26.984612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08734680-efcd-4523-ba48-96dd18a64312 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:26.985207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08734680-efcd-4523-ba48-96dd18a64312 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:26.985477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08734680-efcd-4523-ba48-96dd18a64312 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:26.985748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08734680-efcd-4523-ba48-96dd18a64312 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:27.010185 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=100 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:27.051051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08734680-efcd-4523-ba48-96dd18a64312 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:27.052953 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08734680-efcd-4523-ba48-96dd18a64312 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:27.063456 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1389/2748] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:28.301721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc77892-b9d2-4bad-aaf9-4da123e1be24 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:28.302189 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc77892-b9d2-4bad-aaf9-4da123e1be24 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:28.302506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc77892-b9d2-4bad-aaf9-4da123e1be24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:28.328632 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-9fc77892-b9d2-4bad-aaf9-4da123e1be24 None None] MFA Rules not processed for user `2df1b8bc5012478b9ac5b4c2341a8d12`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:28.329941 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9fc77892-b9d2-4bad-aaf9-4da123e1be24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:28.342588 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1360/2749] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:29.857606 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08734680-efcd-4523-ba48-96dd18a64312 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:29.930857 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08734680-efcd-4523-ba48-96dd18a64312 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:29.936768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:29.937447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:29.937723 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:29.937992 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] PATH_INFO: `/v3/users/8b4cfeab438c41b7914ea577649c0aae` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:29.939833 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b4cfeab438c41b7914ea577649c0aae)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:29.941117 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:29.999740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b4cfeab438c41b7914ea577649c0aae'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.021308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b4cfeab438c41b7914ea577649c0aae'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.030792 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b4cfeab438c41b7914ea577649c0aae'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.031729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b4cfeab438c41b7914ea577649c0aae'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.032412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b4cfeab438c41b7914ea577649c0aae'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.051100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1390/2750] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:29 2022] DELETE /identity/v3/users/8b4cfeab438c41b7914ea577649c0aae => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:30.056166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd4a78-73d4-406e-bd4d-863673ae9942 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:30.056692 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd4a78-73d4-406e-bd4d-863673ae9942 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:30.056792 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd4a78-73d4-406e-bd4d-863673ae9942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:30.077801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-a8bd4a78-73d4-406e-bd4d-863673ae9942 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:30.079348 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a8bd4a78-73d4-406e-bd4d-863673ae9942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:30.143989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1361/2751] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:30.154434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:30.155927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:30.207784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:30.223024 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6e801-0b15-4206-8e24-adcc593404d3 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:30.227100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401af975-1e03-4144-ab32-52cf2c6828c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:30.227735 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401af975-1e03-4144-ab32-52cf2c6828c1 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:30.228011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401af975-1e03-4144-ab32-52cf2c6828c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:30.228309 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401af975-1e03-4144-ab32-52cf2c6828c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:30.238774 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401af975-1e03-4144-ab32-52cf2c6828c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:30.240177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401af975-1e03-4144-ab32-52cf2c6828c1 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:30.250831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1391/2752] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:30.540121 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd4a78-73d4-406e-bd4d-863673ae9942 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:30.541675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a8bd4a78-73d4-406e-bd4d-863673ae9942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:30.556898 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd4a78-73d4-406e-bd4d-863673ae9942 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:30.562082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:30.563197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:30.563197 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:30.563506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] PATH_INFO: `/v3/projects/372345642a564430b9b41421ffde0afa` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:30.565026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=372345642a564430b9b41421ffde0afa)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:30.567271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:30.578845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:30.623222 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '372345642a564430b9b41421ffde0afa', 'request_id': 'req-eb8d3875-ba17-4558-b67d-9b68778193d3'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.629565 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '372345642a564430b9b41421ffde0afa', 'request_id': 'req-eb8d3875-ba17-4558-b67d-9b68778193d3'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.638089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '372345642a564430b9b41421ffde0afa', 'request_id': 'req-eb8d3875-ba17-4558-b67d-9b68778193d3'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.648922 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] The token cache is being invalidate because project 372345642a564430b9b41421ffde0afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:30.649276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.649933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.651812 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1362/2753] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:30 2022] DELETE /identity/v3/projects/372345642a564430b9b41421ffde0afa => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:30.655828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401af975-1e03-4144-ab32-52cf2c6828c1 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:30.667821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401af975-1e03-4144-ab32-52cf2c6828c1 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:30.728088 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6c251-5f98-4275-b59b-6887ca39dad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:30.729008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6c251-5f98-4275-b59b-6887ca39dad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:30.729288 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6c251-5f98-4275-b59b-6887ca39dad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:30.729583 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6c251-5f98-4275-b59b-6887ca39dad2 admin admin] PATH_INFO: `/v3/users/a7d70a286c6348d3adec201d22af811a` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:30.730787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6c251-5f98-4275-b59b-6887ca39dad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d70a286c6348d3adec201d22af811a)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:30.732735 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6c251-5f98-4275-b59b-6887ca39dad2 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:30.783027 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-0dd6c251-5f98-4275-b59b-6887ca39dad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d70a286c6348d3adec201d22af811a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.796692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-0dd6c251-5f98-4275-b59b-6887ca39dad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d70a286c6348d3adec201d22af811a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.804894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-0dd6c251-5f98-4275-b59b-6887ca39dad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d70a286c6348d3adec201d22af811a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.805997 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-0dd6c251-5f98-4275-b59b-6887ca39dad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d70a286c6348d3adec201d22af811a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.806922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-0dd6c251-5f98-4275-b59b-6887ca39dad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d70a286c6348d3adec201d22af811a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:30.820860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1392/2754] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:30 2022] DELETE /identity/v3/users/a7d70a286c6348d3adec201d22af811a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:31.125286 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:31.127510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:31.150259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d3875-ba17-4558-b67d-9b68778193d3 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:31.215204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65735589-6d31-4699-aeb6-ea870305a460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:31.216194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65735589-6d31-4699-aeb6-ea870305a460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:31.216471 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65735589-6d31-4699-aeb6-ea870305a460 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:31.216746 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65735589-6d31-4699-aeb6-ea870305a460 admin admin] PATH_INFO: `/v3/projects/4739b7d431da4b058fc048cfe74e0537` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:31.218108 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65735589-6d31-4699-aeb6-ea870305a460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4739b7d431da4b058fc048cfe74e0537)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:31.220093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65735589-6d31-4699-aeb6-ea870305a460 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:31.273750 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-65735589-6d31-4699-aeb6-ea870305a460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4739b7d431da4b058fc048cfe74e0537', 'request_id': 'req-65735589-6d31-4699-aeb6-ea870305a460'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:31.282171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-65735589-6d31-4699-aeb6-ea870305a460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4739b7d431da4b058fc048cfe74e0537', 'request_id': 'req-65735589-6d31-4699-aeb6-ea870305a460'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:31.292167 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-65735589-6d31-4699-aeb6-ea870305a460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4739b7d431da4b058fc048cfe74e0537', 'request_id': 'req-65735589-6d31-4699-aeb6-ea870305a460'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:31.306066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-65735589-6d31-4699-aeb6-ea870305a460 admin admin] The token cache is being invalidate because project 4739b7d431da4b058fc048cfe74e0537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:31.306885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-65735589-6d31-4699-aeb6-ea870305a460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:31.307061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-65735589-6d31-4699-aeb6-ea870305a460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:31.308664 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1363/2755] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:31 2022] DELETE /identity/v3/projects/4739b7d431da4b058fc048cfe74e0537 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:31.315640 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427d6ce-c06e-4d55-a0f0-32d65bb672cb None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:31.316214 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427d6ce-c06e-4d55-a0f0-32d65bb672cb None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:31.316675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427d6ce-c06e-4d55-a0f0-32d65bb672cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:31.337186 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-7427d6ce-c06e-4d55-a0f0-32d65bb672cb None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:31.338419 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7427d6ce-c06e-4d55-a0f0-32d65bb672cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:31.416194 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1393/2756] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:31.418061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2321c-2ed8-40ff-8865-5890069b1390 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:31.418415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2321c-2ed8-40ff-8865-5890069b1390 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:31.418791 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2321c-2ed8-40ff-8865-5890069b1390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:31.439945 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-e1c2321c-2ed8-40ff-8865-5890069b1390 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:31.441148 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e1c2321c-2ed8-40ff-8865-5890069b1390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:31.449595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1364/2757] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:31.454062 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427d6ce-c06e-4d55-a0f0-32d65bb672cb None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:31.460155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:31.464574 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7427d6ce-c06e-4d55-a0f0-32d65bb672cb None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:31.470968 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0fabb-f98c-42bf-be5c-62365a965e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:31.471572 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0fabb-f98c-42bf-be5c-62365a965e37 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:31.472077 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0fabb-f98c-42bf-be5c-62365a965e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:31.472813 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0fabb-f98c-42bf-be5c-62365a965e37 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:31.473587 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0fabb-f98c-42bf-be5c-62365a965e37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:58:31.473952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0fabb-f98c-42bf-be5c-62365a965e37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:31.475967 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0fabb-f98c-42bf-be5c-62365a965e37 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:31.489529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1394/2758] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:58:31.494965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2321c-2ed8-40ff-8865-5890069b1390 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:31.504509 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2321c-2ed8-40ff-8865-5890069b1390 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:31.510140 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0c3a4-8282-4936-ace8-7b6e4241a954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:31.510793 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0c3a4-8282-4936-ace8-7b6e4241a954 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:31.511070 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0c3a4-8282-4936-ace8-7b6e4241a954 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:31.511367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0c3a4-8282-4936-ace8-7b6e4241a954 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:31.512026 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0c3a4-8282-4936-ace8-7b6e4241a954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:31.513921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0c3a4-8282-4936-ace8-7b6e4241a954 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:31.537521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:31.549063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1365/2759] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:31 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:31.550192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0fabb-f98c-42bf-be5c-62365a965e37 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:31.561094 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0fabb-f98c-42bf-be5c-62365a965e37 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:31.619536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54859af-3357-43eb-bdbc-72bf1c6b1039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:31.620153 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54859af-3357-43eb-bdbc-72bf1c6b1039 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:31.620426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54859af-3357-43eb-bdbc-72bf1c6b1039 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:31.620690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54859af-3357-43eb-bdbc-72bf1c6b1039 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:31.621470 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54859af-3357-43eb-bdbc-72bf1c6b1039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:31.623915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54859af-3357-43eb-bdbc-72bf1c6b1039 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:31.645708 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1395/2760] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:31 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:31.650116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0c3a4-8282-4936-ace8-7b6e4241a954 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:31.660676 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0c3a4-8282-4936-ace8-7b6e4241a954 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:31.666503 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fa4cb-0224-4521-bec5-05112702da19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:31.667161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fa4cb-0224-4521-bec5-05112702da19 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:31.667440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fa4cb-0224-4521-bec5-05112702da19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:31.667710 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fa4cb-0224-4521-bec5-05112702da19 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:31.668400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fa4cb-0224-4521-bec5-05112702da19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:31.670006 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fa4cb-0224-4521-bec5-05112702da19 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:31.680361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1366/2761] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:31.684678 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54859af-3357-43eb-bdbc-72bf1c6b1039 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:31.694687 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54859af-3357-43eb-bdbc-72bf1c6b1039 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:31.699637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51e4bf-1c98-48c8-9603-98dc0f347a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:31.700252 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51e4bf-1c98-48c8-9603-98dc0f347a56 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:31.700534 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51e4bf-1c98-48c8-9603-98dc0f347a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:31.700808 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51e4bf-1c98-48c8-9603-98dc0f347a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:31.701503 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51e4bf-1c98-48c8-9603-98dc0f347a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:31.703155 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51e4bf-1c98-48c8-9603-98dc0f347a56 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:31.714552 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1396/2762] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:31.718696 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fa4cb-0224-4521-bec5-05112702da19 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:31.729228 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fa4cb-0224-4521-bec5-05112702da19 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:31.735349 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68143678-a9cc-4fc4-8a81-eb1c46798875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:31.736260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68143678-a9cc-4fc4-8a81-eb1c46798875 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:31.736550 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68143678-a9cc-4fc4-8a81-eb1c46798875 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:31.736821 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68143678-a9cc-4fc4-8a81-eb1c46798875 admin admin] PATH_INFO: `/v3/projects/0a83a2e6f87249279a1e953474662f8a/users/047ade3407a642f3b1618400ed7edcc7/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:31.756721 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68143678-a9cc-4fc4-8a81-eb1c46798875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a83a2e6f87249279a1e953474662f8a, user_id=047ade3407a642f3b1618400ed7edcc7, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:31.759831 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68143678-a9cc-4fc4-8a81-eb1c46798875 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:31.774115 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1367/2763] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:31 2022] PUT /identity/v3/projects/0a83a2e6f87249279a1e953474662f8a/users/047ade3407a642f3b1618400ed7edcc7/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:31.785499 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e71fc-a382-4b8f-9a85-4e066953f02f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:31.785784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e71fc-a382-4b8f-9a85-4e066953f02f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:31.786051 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e71fc-a382-4b8f-9a85-4e066953f02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:31.814887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-b75e71fc-a382-4b8f-9a85-4e066953f02f None None] MFA Rules not processed for user `047ade3407a642f3b1618400ed7edcc7`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:31.816013 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b75e71fc-a382-4b8f-9a85-4e066953f02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:31.881292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1397/2764] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:31.886301 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68143678-a9cc-4fc4-8a81-eb1c46798875 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:31.949514 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68143678-a9cc-4fc4-8a81-eb1c46798875 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:31.954251 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee1770-27e6-44de-be42-a146605bc93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:31.954877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee1770-27e6-44de-be42-a146605bc93c admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:31.955150 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee1770-27e6-44de-be42-a146605bc93c admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:31.955420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee1770-27e6-44de-be42-a146605bc93c admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:31.956095 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee1770-27e6-44de-be42-a146605bc93c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:31.957955 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee1770-27e6-44de-be42-a146605bc93c admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:31.984921 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1368/2765] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:31 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:31.989128 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e71fc-a382-4b8f-9a85-4e066953f02f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:32.000076 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e71fc-a382-4b8f-9a85-4e066953f02f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:32.058917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46b651-41d7-4bf0-b6c1-4dc0cac6e44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:32.059536 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46b651-41d7-4bf0-b6c1-4dc0cac6e44e admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:32.059812 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46b651-41d7-4bf0-b6c1-4dc0cac6e44e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:32.060080 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46b651-41d7-4bf0-b6c1-4dc0cac6e44e admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:32.060729 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46b651-41d7-4bf0-b6c1-4dc0cac6e44e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:32.062647 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46b651-41d7-4bf0-b6c1-4dc0cac6e44e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:32.071066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:32.084929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1398/2766] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:31 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:32.089986 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee1770-27e6-44de-be42-a146605bc93c admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:32.100419 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee1770-27e6-44de-be42-a146605bc93c admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:32.105449 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ff13a-953a-425e-8033-dc04ec909e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:32.106072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ff13a-953a-425e-8033-dc04ec909e4a admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:32.106333 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ff13a-953a-425e-8033-dc04ec909e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:32.106607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ff13a-953a-425e-8033-dc04ec909e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:32.107319 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ff13a-953a-425e-8033-dc04ec909e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:32.108905 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ff13a-953a-425e-8033-dc04ec909e4a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:32.119429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1369/2767] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:32.123381 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46b651-41d7-4bf0-b6c1-4dc0cac6e44e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:32.134325 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46b651-41d7-4bf0-b6c1-4dc0cac6e44e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:32.140593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc247e-aa55-4811-8607-ade071adf095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:32.141251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc247e-aa55-4811-8607-ade071adf095 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:32.141529 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc247e-aa55-4811-8607-ade071adf095 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:32.141802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc247e-aa55-4811-8607-ade071adf095 admin admin] PATH_INFO: `/v3/projects/4f15266513304124971faa200ff8e719/users/a577e2fbbcb0461495a61de4976dc2cb/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:32.157401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc247e-aa55-4811-8607-ade071adf095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f15266513304124971faa200ff8e719, user_id=a577e2fbbcb0461495a61de4976dc2cb, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:32.160100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc247e-aa55-4811-8607-ade071adf095 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:32.172934 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1399/2768] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:32 2022] PUT /identity/v3/projects/4f15266513304124971faa200ff8e719/users/a577e2fbbcb0461495a61de4976dc2cb/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:32.176849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ff13a-953a-425e-8033-dc04ec909e4a admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:32.187816 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ff13a-953a-425e-8033-dc04ec909e4a admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:32.243539 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fe1fa-b2ea-4595-afd5-70c835bce278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:32.244182 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fe1fa-b2ea-4595-afd5-70c835bce278 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:32.244468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fe1fa-b2ea-4595-afd5-70c835bce278 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:32.244740 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fe1fa-b2ea-4595-afd5-70c835bce278 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:32.245415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fe1fa-b2ea-4595-afd5-70c835bce278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:32.247092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fe1fa-b2ea-4595-afd5-70c835bce278 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:32.256734 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1370/2769] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:32.261466 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc247e-aa55-4811-8607-ade071adf095 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:32.271721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc247e-aa55-4811-8607-ade071adf095 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:32.277075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe5589-6bfb-4884-ad68-8be2e062b22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:32.277731 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe5589-6bfb-4884-ad68-8be2e062b22e admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:32.278005 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe5589-6bfb-4884-ad68-8be2e062b22e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:32.278275 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe5589-6bfb-4884-ad68-8be2e062b22e admin admin] PATH_INFO: `/v3/projects/4f15266513304124971faa200ff8e719/users/a577e2fbbcb0461495a61de4976dc2cb/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:32.280245 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe5589-6bfb-4884-ad68-8be2e062b22e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f15266513304124971faa200ff8e719, user_id=a577e2fbbcb0461495a61de4976dc2cb, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:32.282905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe5589-6bfb-4884-ad68-8be2e062b22e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:32.294692 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1400/2770] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:32 2022] PUT /identity/v3/projects/4f15266513304124971faa200ff8e719/users/a577e2fbbcb0461495a61de4976dc2cb/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:32.305223 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eab9d7-126a-487a-b359-158bdc5ef647 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:32.305541 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eab9d7-126a-487a-b359-158bdc5ef647 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:32.306066 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eab9d7-126a-487a-b359-158bdc5ef647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:32.334406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-a3eab9d7-126a-487a-b359-158bdc5ef647 None None] MFA Rules not processed for user `a577e2fbbcb0461495a61de4976dc2cb`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:32.335574 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a3eab9d7-126a-487a-b359-158bdc5ef647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:32.359039 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:32.406264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1371/2771] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:32.417925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe5589-6bfb-4884-ad68-8be2e062b22e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:32.419740 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-1ffe5589-6bfb-4884-ad68-8be2e062b22e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:32.480592 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe5589-6bfb-4884-ad68-8be2e062b22e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:32.485101 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0a6ff-816c-4120-9b5a-8393677306a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:32.485836 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0a6ff-816c-4120-9b5a-8393677306a8 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:32.486241 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0a6ff-816c-4120-9b5a-8393677306a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:32.486710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0a6ff-816c-4120-9b5a-8393677306a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:32.497876 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0a6ff-816c-4120-9b5a-8393677306a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:32.499412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0a6ff-816c-4120-9b5a-8393677306a8 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:32.512673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1401/2772] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:32.558822 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eab9d7-126a-487a-b359-158bdc5ef647 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:32.560268 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a3eab9d7-126a-487a-b359-158bdc5ef647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:32.573573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eab9d7-126a-487a-b359-158bdc5ef647 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:32.580009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b2d57-5db8-4d2f-9f37-91e1fde6c82d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:32.580646 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b2d57-5db8-4d2f-9f37-91e1fde6c82d service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:32.580929 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b2d57-5db8-4d2f-9f37-91e1fde6c82d service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:32.581205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b2d57-5db8-4d2f-9f37-91e1fde6c82d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:32.644447 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b2d57-5db8-4d2f-9f37-91e1fde6c82d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:32.645966 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b2d57-5db8-4d2f-9f37-91e1fde6c82d service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:32.655968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1372/2773] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:36.054701 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0a6ff-816c-4120-9b5a-8393677306a8 service cinder] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:36.119452 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0a6ff-816c-4120-9b5a-8393677306a8 service cinder] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:36.123995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:36.124765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:36.125038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:36.125304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] PATH_INFO: `/v3/users/047ade3407a642f3b1618400ed7edcc7` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:36.126790 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=047ade3407a642f3b1618400ed7edcc7)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:36.128724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:36.185271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '047ade3407a642f3b1618400ed7edcc7'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.192184 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:INSERT=5,keystone:SELECT=87,keystone:DELETE=8 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:36.198317 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '047ade3407a642f3b1618400ed7edcc7'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.206641 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '047ade3407a642f3b1618400ed7edcc7'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.207251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '047ade3407a642f3b1618400ed7edcc7'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.207845 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '047ade3407a642f3b1618400ed7edcc7'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.220815 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1402/2774] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:36 2022] DELETE /identity/v3/users/047ade3407a642f3b1618400ed7edcc7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:36.225351 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff03e9-8f22-4554-b95c-8e79a0b70b04 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:36.225848 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff03e9-8f22-4554-b95c-8e79a0b70b04 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:36.226041 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff03e9-8f22-4554-b95c-8e79a0b70b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:36.246529 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-62ff03e9-8f22-4554-b95c-8e79a0b70b04 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:36.247974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-62ff03e9-8f22-4554-b95c-8e79a0b70b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:36.306282 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1373/2775] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:36.315891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:36.317314 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:36.374758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76837f50-b245-4c98-b888-2393fd844490 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:36.378443 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888974b-f62b-4a50-a934-d7cb52ed6efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:36.379121 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888974b-f62b-4a50-a934-d7cb52ed6efa service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:36.379401 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888974b-f62b-4a50-a934-d7cb52ed6efa service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:36.379674 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888974b-f62b-4a50-a934-d7cb52ed6efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:36.389107 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888974b-f62b-4a50-a934-d7cb52ed6efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:36.390612 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888974b-f62b-4a50-a934-d7cb52ed6efa service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:36.400814 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1403/2776] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:36.688606 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff03e9-8f22-4554-b95c-8e79a0b70b04 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:36.690187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-62ff03e9-8f22-4554-b95c-8e79a0b70b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:36.704613 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff03e9-8f22-4554-b95c-8e79a0b70b04 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:36.709269 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:36.709933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:36.710203 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:36.710471 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] PATH_INFO: `/v3/projects/0a83a2e6f87249279a1e953474662f8a` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:36.712504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a83a2e6f87249279a1e953474662f8a)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:36.714490 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:36.763152 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a83a2e6f87249279a1e953474662f8a', 'request_id': 'req-042ad67b-6eaf-4462-a4f0-3b059bc42c67'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.769353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a83a2e6f87249279a1e953474662f8a', 'request_id': 'req-042ad67b-6eaf-4462-a4f0-3b059bc42c67'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.771034 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:36.779024 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a83a2e6f87249279a1e953474662f8a', 'request_id': 'req-042ad67b-6eaf-4462-a4f0-3b059bc42c67'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.791801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] The token cache is being invalidate because project 0a83a2e6f87249279a1e953474662f8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:36.792089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.792770 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.794404 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1374/2777] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:36 2022] DELETE /identity/v3/projects/0a83a2e6f87249279a1e953474662f8a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:36.798835 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888974b-f62b-4a50-a934-d7cb52ed6efa service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:36.809773 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888974b-f62b-4a50-a934-d7cb52ed6efa service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:36.869741 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9a019-4ffd-4590-a1a5-696f1045de88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:36.870546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9a019-4ffd-4590-a1a5-696f1045de88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:36.871010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9a019-4ffd-4590-a1a5-696f1045de88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:36.871409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9a019-4ffd-4590-a1a5-696f1045de88 admin admin] PATH_INFO: `/v3/users/a577e2fbbcb0461495a61de4976dc2cb` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:36.872676 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9a019-4ffd-4590-a1a5-696f1045de88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a577e2fbbcb0461495a61de4976dc2cb)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:36.874650 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9a019-4ffd-4590-a1a5-696f1045de88 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:36.934828 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a4a9a019-4ffd-4590-a1a5-696f1045de88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a577e2fbbcb0461495a61de4976dc2cb'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.952663 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a4a9a019-4ffd-4590-a1a5-696f1045de88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a577e2fbbcb0461495a61de4976dc2cb'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.962592 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a4a9a019-4ffd-4590-a1a5-696f1045de88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a577e2fbbcb0461495a61de4976dc2cb'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.963562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a4a9a019-4ffd-4590-a1a5-696f1045de88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a577e2fbbcb0461495a61de4976dc2cb'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.964021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a4a9a019-4ffd-4590-a1a5-696f1045de88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a577e2fbbcb0461495a61de4976dc2cb'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:36.970748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:36.987747 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1404/2778] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:36 2022] DELETE /identity/v3/users/a577e2fbbcb0461495a61de4976dc2cb => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:37.287161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:37.288823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:37.304362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ad67b-6eaf-4462-a4f0-3b059bc42c67 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:37.383589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64238d3-4474-43e8-a6ca-9a833988e755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:37.383589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64238d3-4474-43e8-a6ca-9a833988e755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:37.383923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64238d3-4474-43e8-a6ca-9a833988e755 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:37.384138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64238d3-4474-43e8-a6ca-9a833988e755 admin admin] PATH_INFO: `/v3/projects/4f15266513304124971faa200ff8e719` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:37.385463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64238d3-4474-43e8-a6ca-9a833988e755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f15266513304124971faa200ff8e719)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:37.387709 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64238d3-4474-43e8-a6ca-9a833988e755 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:37.438262 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c64238d3-4474-43e8-a6ca-9a833988e755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f15266513304124971faa200ff8e719', 'request_id': 'req-c64238d3-4474-43e8-a6ca-9a833988e755'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:37.444358 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c64238d3-4474-43e8-a6ca-9a833988e755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f15266513304124971faa200ff8e719', 'request_id': 'req-c64238d3-4474-43e8-a6ca-9a833988e755'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:37.453110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c64238d3-4474-43e8-a6ca-9a833988e755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f15266513304124971faa200ff8e719', 'request_id': 'req-c64238d3-4474-43e8-a6ca-9a833988e755'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:37.463627 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c64238d3-4474-43e8-a6ca-9a833988e755 admin admin] The token cache is being invalidate because project 4f15266513304124971faa200ff8e719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:37.463913 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c64238d3-4474-43e8-a6ca-9a833988e755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:37.464530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c64238d3-4474-43e8-a6ca-9a833988e755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:37.466028 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1375/2779] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:37 2022] DELETE /identity/v3/projects/4f15266513304124971faa200ff8e719 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:37.471055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a858c0f-4003-468e-9323-3b7afc56058f None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:37.471528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a858c0f-4003-468e-9323-3b7afc56058f None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:37.471626 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a858c0f-4003-468e-9323-3b7afc56058f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:37.492891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-5a858c0f-4003-468e-9323-3b7afc56058f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:37.494286 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-5a858c0f-4003-468e-9323-3b7afc56058f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:37.554921 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1405/2780] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:37.564376 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7003ef1-e307-4387-b0c0-166ccedf308f None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:37.564658 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7003ef1-e307-4387-b0c0-166ccedf308f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:37.564923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7003ef1-e307-4387-b0c0-166ccedf308f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:37.584716 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-f7003ef1-e307-4387-b0c0-166ccedf308f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:37.586394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f7003ef1-e307-4387-b0c0-166ccedf308f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:37.594558 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1376/2781] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:37.598928 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a858c0f-4003-468e-9323-3b7afc56058f None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:37.609597 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a858c0f-4003-468e-9323-3b7afc56058f None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:37.614170 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366fd2f-6c62-42b4-9b61-17beea41b92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:37.614814 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366fd2f-6c62-42b4-9b61-17beea41b92b admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:37.615097 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366fd2f-6c62-42b4-9b61-17beea41b92b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:37.615372 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366fd2f-6c62-42b4-9b61-17beea41b92b admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:37.616061 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366fd2f-6c62-42b4-9b61-17beea41b92b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:58:37.616364 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366fd2f-6c62-42b4-9b61-17beea41b92b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:37.618075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366fd2f-6c62-42b4-9b61-17beea41b92b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:37.629505 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1406/2782] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:58:37.634360 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7003ef1-e307-4387-b0c0-166ccedf308f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:37.643631 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7003ef1-e307-4387-b0c0-166ccedf308f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:37.648195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271daf06-c0a4-478a-806a-1d7516a91dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:37.649727 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271daf06-c0a4-478a-806a-1d7516a91dea admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:37.650003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271daf06-c0a4-478a-806a-1d7516a91dea admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:37.650271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271daf06-c0a4-478a-806a-1d7516a91dea admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:37.650968 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271daf06-c0a4-478a-806a-1d7516a91dea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:37.652856 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271daf06-c0a4-478a-806a-1d7516a91dea admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:37.681666 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1377/2783] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:37 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:58:37.686018 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366fd2f-6c62-42b4-9b61-17beea41b92b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:37.696042 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366fd2f-6c62-42b4-9b61-17beea41b92b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:37.754134 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb174e3d-a3cc-4a41-9ccf-a5503c9bbceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:37.754779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb174e3d-a3cc-4a41-9ccf-a5503c9bbceb admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:37.755056 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb174e3d-a3cc-4a41-9ccf-a5503c9bbceb admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:37.755344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb174e3d-a3cc-4a41-9ccf-a5503c9bbceb admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:37.755991 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb174e3d-a3cc-4a41-9ccf-a5503c9bbceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:37.758228 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb174e3d-a3cc-4a41-9ccf-a5503c9bbceb admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:37.780261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1407/2784] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:37 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:37.784236 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271daf06-c0a4-478a-806a-1d7516a91dea admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:37.795464 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271daf06-c0a4-478a-806a-1d7516a91dea admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:37.800102 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a617b7-ed11-45e4-970f-44976c3c58e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:37.800708 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a617b7-ed11-45e4-970f-44976c3c58e7 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:37.801107 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a617b7-ed11-45e4-970f-44976c3c58e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:37.801374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a617b7-ed11-45e4-970f-44976c3c58e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:37.802061 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a617b7-ed11-45e4-970f-44976c3c58e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:37.803664 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a617b7-ed11-45e4-970f-44976c3c58e7 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:37.813996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1378/2785] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:37.817943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb174e3d-a3cc-4a41-9ccf-a5503c9bbceb admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:37.828634 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb174e3d-a3cc-4a41-9ccf-a5503c9bbceb admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:37.833247 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6f940-de23-4af5-acc5-6811742aab55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:37.833841 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6f940-de23-4af5-acc5-6811742aab55 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:37.834116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6f940-de23-4af5-acc5-6811742aab55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:37.834399 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6f940-de23-4af5-acc5-6811742aab55 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:37.835111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6f940-de23-4af5-acc5-6811742aab55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:37.837710 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6f940-de23-4af5-acc5-6811742aab55 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:37.849248 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1408/2786] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:37.854480 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a617b7-ed11-45e4-970f-44976c3c58e7 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:37.864175 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a617b7-ed11-45e4-970f-44976c3c58e7 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:37.869168 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1ebef-6b6e-4307-b16e-7108d10db849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:37.869808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1ebef-6b6e-4307-b16e-7108d10db849 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:37.870068 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1ebef-6b6e-4307-b16e-7108d10db849 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:37.870336 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1ebef-6b6e-4307-b16e-7108d10db849 admin admin] PATH_INFO: `/v3/projects/7648d50f0cb14b83b55e9eee0fd805e1/users/2a74095d7ded411a9a31a4718b73fbab/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:37.884540 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1ebef-6b6e-4307-b16e-7108d10db849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7648d50f0cb14b83b55e9eee0fd805e1, user_id=2a74095d7ded411a9a31a4718b73fbab, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:37.887123 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1ebef-6b6e-4307-b16e-7108d10db849 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:37.900607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1379/2787] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:37 2022] PUT /identity/v3/projects/7648d50f0cb14b83b55e9eee0fd805e1/users/2a74095d7ded411a9a31a4718b73fbab/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:37.910783 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ece84-9005-4134-9ec8-1dcf0886407a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:37.911069 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ece84-9005-4134-9ec8-1dcf0886407a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:37.911338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ece84-9005-4134-9ec8-1dcf0886407a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:37.939089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-080ece84-9005-4134-9ec8-1dcf0886407a None None] MFA Rules not processed for user `2a74095d7ded411a9a31a4718b73fbab`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:37.940289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-080ece84-9005-4134-9ec8-1dcf0886407a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:37.987821 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:38.004985 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1409/2788] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:38.010104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1ebef-6b6e-4307-b16e-7108d10db849 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:38.020093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:38.076802 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1ebef-6b6e-4307-b16e-7108d10db849 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:38.081544 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa8d92-777d-4000-a86a-e6017a4b6276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:38.082152 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa8d92-777d-4000-a86a-e6017a4b6276 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:38.082430 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa8d92-777d-4000-a86a-e6017a4b6276 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:38.082739 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa8d92-777d-4000-a86a-e6017a4b6276 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:38.083452 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa8d92-777d-4000-a86a-e6017a4b6276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:38.085322 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa8d92-777d-4000-a86a-e6017a4b6276 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:38.112135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1380/2789] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:38 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:38.116284 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ece84-9005-4134-9ec8-1dcf0886407a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:38.127622 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ece84-9005-4134-9ec8-1dcf0886407a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:38.185251 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46081e2-116c-4a68-911e-271f260a1d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:38.185869 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46081e2-116c-4a68-911e-271f260a1d69 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:38.186146 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46081e2-116c-4a68-911e-271f260a1d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:38.186418 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46081e2-116c-4a68-911e-271f260a1d69 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:38.187116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46081e2-116c-4a68-911e-271f260a1d69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:38.189030 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46081e2-116c-4a68-911e-271f260a1d69 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:38.210297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1410/2790] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:38 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:38.215091 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa8d92-777d-4000-a86a-e6017a4b6276 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:38.225459 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa8d92-777d-4000-a86a-e6017a4b6276 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:38.231967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc162a-f30d-4295-b3fb-205f6c83e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:38.232612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc162a-f30d-4295-b3fb-205f6c83e471 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:38.232888 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc162a-f30d-4295-b3fb-205f6c83e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:38.233156 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc162a-f30d-4295-b3fb-205f6c83e471 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:38.233845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc162a-f30d-4295-b3fb-205f6c83e471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:38.235530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc162a-f30d-4295-b3fb-205f6c83e471 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:38.247826 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1381/2791] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:38.252144 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46081e2-116c-4a68-911e-271f260a1d69 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:38.264525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46081e2-116c-4a68-911e-271f260a1d69 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:38.269623 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fe4f4-bbb2-4b21-b07a-b9740e6f395b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:38.270290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fe4f4-bbb2-4b21-b07a-b9740e6f395b admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:38.270561 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fe4f4-bbb2-4b21-b07a-b9740e6f395b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:38.270859 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fe4f4-bbb2-4b21-b07a-b9740e6f395b admin admin] PATH_INFO: `/v3/projects/65ed54ec19d540009e456544ebf63271/users/bed331249b6d41ec997e3d97031a83fc/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:38.287376 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526fe4f4-bbb2-4b21-b07a-b9740e6f395b admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ed54ec19d540009e456544ebf63271, user_id=bed331249b6d41ec997e3d97031a83fc, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:38.291153 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526fe4f4-bbb2-4b21-b07a-b9740e6f395b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:38.304406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1411/2792] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:38 2022] PUT /identity/v3/projects/65ed54ec19d540009e456544ebf63271/users/bed331249b6d41ec997e3d97031a83fc/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:38.308778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc162a-f30d-4295-b3fb-205f6c83e471 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:38.318664 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc162a-f30d-4295-b3fb-205f6c83e471 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:38.381631 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6251529-c768-4b32-b8d7-25ab8e13e228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:38.382246 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6251529-c768-4b32-b8d7-25ab8e13e228 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:38.382521 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6251529-c768-4b32-b8d7-25ab8e13e228 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:38.382890 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6251529-c768-4b32-b8d7-25ab8e13e228 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:38.384205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6251529-c768-4b32-b8d7-25ab8e13e228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:38.386406 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6251529-c768-4b32-b8d7-25ab8e13e228 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:38.399800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1382/2793] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:38.402386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fe4f4-bbb2-4b21-b07a-b9740e6f395b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:38.413038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fe4f4-bbb2-4b21-b07a-b9740e6f395b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:38.419038 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841ecca-3078-43bb-95ca-14edbb30ef24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:38.419770 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841ecca-3078-43bb-95ca-14edbb30ef24 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:38.420118 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841ecca-3078-43bb-95ca-14edbb30ef24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:38.420449 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841ecca-3078-43bb-95ca-14edbb30ef24 admin admin] PATH_INFO: `/v3/projects/65ed54ec19d540009e456544ebf63271/users/bed331249b6d41ec997e3d97031a83fc/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:38.422477 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841ecca-3078-43bb-95ca-14edbb30ef24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ed54ec19d540009e456544ebf63271, user_id=bed331249b6d41ec997e3d97031a83fc, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:38.425429 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841ecca-3078-43bb-95ca-14edbb30ef24 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:38.436631 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1412/2794] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:38 2022] PUT /identity/v3/projects/65ed54ec19d540009e456544ebf63271/users/bed331249b6d41ec997e3d97031a83fc/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:38.446886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13507cd-1bfe-4a9a-8685-9364769ff3d4 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:38.447177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13507cd-1bfe-4a9a-8685-9364769ff3d4 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:38.447451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13507cd-1bfe-4a9a-8685-9364769ff3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:38.478414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-c13507cd-1bfe-4a9a-8685-9364769ff3d4 None None] MFA Rules not processed for user `bed331249b6d41ec997e3d97031a83fc`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:38.479773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c13507cd-1bfe-4a9a-8685-9364769ff3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:38.554205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1383/2795] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:38.566917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841ecca-3078-43bb-95ca-14edbb30ef24 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:38.568772 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a841ecca-3078-43bb-95ca-14edbb30ef24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:38.628096 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841ecca-3078-43bb-95ca-14edbb30ef24 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:38.631914 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c62c8-07ad-4cf9-82dd-201bb8e9f41f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:38.632524 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c62c8-07ad-4cf9-82dd-201bb8e9f41f service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:38.632834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c62c8-07ad-4cf9-82dd-201bb8e9f41f service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:38.633702 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c62c8-07ad-4cf9-82dd-201bb8e9f41f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:38.643539 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c62c8-07ad-4cf9-82dd-201bb8e9f41f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:38.645397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c62c8-07ad-4cf9-82dd-201bb8e9f41f service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:38.656379 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1413/2796] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:38.810143 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13507cd-1bfe-4a9a-8685-9364769ff3d4 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:38.816206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:38.872711 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13507cd-1bfe-4a9a-8685-9364769ff3d4 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:38.877224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:38.877853 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:38.878125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:38.878394 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] PATH_INFO: `/v3/users/2a74095d7ded411a9a31a4718b73fbab` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:38.879573 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a74095d7ded411a9a31a4718b73fbab)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:38.881466 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:38.932885 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a74095d7ded411a9a31a4718b73fbab'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:38.941533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a74095d7ded411a9a31a4718b73fbab'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:38.942211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a74095d7ded411a9a31a4718b73fbab'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:38.956961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a74095d7ded411a9a31a4718b73fbab'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:38.966504 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a74095d7ded411a9a31a4718b73fbab'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:38.973420 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1384/2797] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:38 2022] DELETE /identity/v3/users/2a74095d7ded411a9a31a4718b73fbab => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:38.978287 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3356c0-911c-4d35-b6e8-c1478e2745b4 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:38.978762 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3356c0-911c-4d35-b6e8-c1478e2745b4 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:38.979177 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3356c0-911c-4d35-b6e8-c1478e2745b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:38.999459 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-8a3356c0-911c-4d35-b6e8-c1478e2745b4 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:39.000715 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8a3356c0-911c-4d35-b6e8-c1478e2745b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:39.049416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:39.069546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1414/2798] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:39.080909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:39.082611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:39.150258 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114b729-be73-4506-be0b-002ff6a68dcb admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:39.154182 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f2792-10fd-449c-b7ae-1c7a685469c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:39.154886 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f2792-10fd-449c-b7ae-1c7a685469c7 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:39.155098 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f2792-10fd-449c-b7ae-1c7a685469c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:39.155374 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f2792-10fd-449c-b7ae-1c7a685469c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:39.166092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f2792-10fd-449c-b7ae-1c7a685469c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:39.167593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f2792-10fd-449c-b7ae-1c7a685469c7 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:39.178002 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1385/2799] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:39.569804 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3356c0-911c-4d35-b6e8-c1478e2745b4 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:39.571816 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-8a3356c0-911c-4d35-b6e8-c1478e2745b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:39.588959 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3356c0-911c-4d35-b6e8-c1478e2745b4 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:39.603032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:39.603717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:39.603997 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:39.604265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] PATH_INFO: `/v3/projects/7648d50f0cb14b83b55e9eee0fd805e1` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:39.605457 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7648d50f0cb14b83b55e9eee0fd805e1)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:39.607398 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:39.654451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7648d50f0cb14b83b55e9eee0fd805e1', 'request_id': 'req-a70bd71a-b246-48a3-b6fc-846512c4e931'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:39.660596 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7648d50f0cb14b83b55e9eee0fd805e1', 'request_id': 'req-a70bd71a-b246-48a3-b6fc-846512c4e931'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:39.669275 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7648d50f0cb14b83b55e9eee0fd805e1', 'request_id': 'req-a70bd71a-b246-48a3-b6fc-846512c4e931'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:39.679546 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] The token cache is being invalidate because project 7648d50f0cb14b83b55e9eee0fd805e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:39.679818 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:39.680447 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:39.682341 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1415/2800] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:39 2022] DELETE /identity/v3/projects/7648d50f0cb14b83b55e9eee0fd805e1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:39.686483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f2792-10fd-449c-b7ae-1c7a685469c7 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:39.698563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f2792-10fd-449c-b7ae-1c7a685469c7 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:39.723728 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:39.757373 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05511b5-2f45-48c8-961c-d7b04a555f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:39.758551 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05511b5-2f45-48c8-961c-d7b04a555f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:39.758881 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05511b5-2f45-48c8-961c-d7b04a555f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:39.759410 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05511b5-2f45-48c8-961c-d7b04a555f7d admin admin] PATH_INFO: `/v3/users/bed331249b6d41ec997e3d97031a83fc` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:39.760309 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05511b5-2f45-48c8-961c-d7b04a555f7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bed331249b6d41ec997e3d97031a83fc)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:39.762328 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05511b5-2f45-48c8-961c-d7b04a555f7d admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:39.807224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-a05511b5-2f45-48c8-961c-d7b04a555f7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bed331249b6d41ec997e3d97031a83fc'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:39.813932 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-a05511b5-2f45-48c8-961c-d7b04a555f7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bed331249b6d41ec997e3d97031a83fc'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:39.814590 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-a05511b5-2f45-48c8-961c-d7b04a555f7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bed331249b6d41ec997e3d97031a83fc'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:39.827181 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-a05511b5-2f45-48c8-961c-d7b04a555f7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bed331249b6d41ec997e3d97031a83fc'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:39.834719 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-a05511b5-2f45-48c8-961c-d7b04a555f7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bed331249b6d41ec997e3d97031a83fc'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:39.840951 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1386/2801] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:39 2022] DELETE /identity/v3/users/bed331249b6d41ec997e3d97031a83fc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:40.120515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:40.122143 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:40.135717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bd71a-b246-48a3-b6fc-846512c4e931 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:40.195824 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f71f1-666a-4683-9245-f84d31931bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:40.196641 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f71f1-666a-4683-9245-f84d31931bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:40.196925 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f71f1-666a-4683-9245-f84d31931bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:40.197278 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f71f1-666a-4683-9245-f84d31931bdd admin admin] PATH_INFO: `/v3/projects/65ed54ec19d540009e456544ebf63271` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:40.198404 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f71f1-666a-4683-9245-f84d31931bdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=65ed54ec19d540009e456544ebf63271)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:40.200420 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f71f1-666a-4683-9245-f84d31931bdd admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:40.249129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c7f71f1-666a-4683-9245-f84d31931bdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65ed54ec19d540009e456544ebf63271', 'request_id': 'req-1c7f71f1-666a-4683-9245-f84d31931bdd'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:40.255596 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c7f71f1-666a-4683-9245-f84d31931bdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65ed54ec19d540009e456544ebf63271', 'request_id': 'req-1c7f71f1-666a-4683-9245-f84d31931bdd'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:40.264258 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c7f71f1-666a-4683-9245-f84d31931bdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65ed54ec19d540009e456544ebf63271', 'request_id': 'req-1c7f71f1-666a-4683-9245-f84d31931bdd'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:40.272282 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:40.277716 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c7f71f1-666a-4683-9245-f84d31931bdd admin admin] The token cache is being invalidate because project 65ed54ec19d540009e456544ebf63271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:40.278230 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c7f71f1-666a-4683-9245-f84d31931bdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:40.279514 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-1c7f71f1-666a-4683-9245-f84d31931bdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:40.284360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1416/2802] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:40 2022] DELETE /identity/v3/projects/65ed54ec19d540009e456544ebf63271 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:40.286074 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056ed61-9319-4c95-b3f8-ca5047acf99f None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:40.286361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056ed61-9319-4c95-b3f8-ca5047acf99f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:40.286675 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056ed61-9319-4c95-b3f8-ca5047acf99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:40.307694 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-7056ed61-9319-4c95-b3f8-ca5047acf99f None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:40.308965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7056ed61-9319-4c95-b3f8-ca5047acf99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:40.367825 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1387/2803] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:40.437686 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddda662-a332-48d1-bbac-b776d487674a None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:40.437969 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddda662-a332-48d1-bbac-b776d487674a None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:40.438321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddda662-a332-48d1-bbac-b776d487674a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:40.462413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-eddda662-a332-48d1-bbac-b776d487674a None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:40.464111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-eddda662-a332-48d1-bbac-b776d487674a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:40.473995 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1417/2804] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:40.479138 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056ed61-9319-4c95-b3f8-ca5047acf99f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:40.488705 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056ed61-9319-4c95-b3f8-ca5047acf99f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:40.496156 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a723cff-3f21-4e54-b416-9575a3d4c830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:40.496860 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a723cff-3f21-4e54-b416-9575a3d4c830 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:40.497204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a723cff-3f21-4e54-b416-9575a3d4c830 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:40.497545 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a723cff-3f21-4e54-b416-9575a3d4c830 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:40.498324 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a723cff-3f21-4e54-b416-9575a3d4c830 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:58:40.498728 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a723cff-3f21-4e54-b416-9575a3d4c830 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:40.500415 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a723cff-3f21-4e54-b416-9575a3d4c830 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:40.513396 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1388/2805] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:58:40.518132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddda662-a332-48d1-bbac-b776d487674a None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:40.528724 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddda662-a332-48d1-bbac-b776d487674a None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:40.533515 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782c3cd-d71b-4d04-816f-cefa5eaf965e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:40.534135 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782c3cd-d71b-4d04-816f-cefa5eaf965e admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:40.534412 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782c3cd-d71b-4d04-816f-cefa5eaf965e admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:40.534713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782c3cd-d71b-4d04-816f-cefa5eaf965e admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:40.535374 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782c3cd-d71b-4d04-816f-cefa5eaf965e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:40.537338 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782c3cd-d71b-4d04-816f-cefa5eaf965e admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:40.570721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1418/2806] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:40 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:40.573978 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a723cff-3f21-4e54-b416-9575a3d4c830 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:40.584744 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a723cff-3f21-4e54-b416-9575a3d4c830 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:40.615915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:40.644974 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d70e1-a8ec-470a-80eb-01edcfe657e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:40.645586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d70e1-a8ec-470a-80eb-01edcfe657e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:40.645864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d70e1-a8ec-470a-80eb-01edcfe657e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:40.646137 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d70e1-a8ec-470a-80eb-01edcfe657e5 admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:40.646891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d70e1-a8ec-470a-80eb-01edcfe657e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:40.649496 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d70e1-a8ec-470a-80eb-01edcfe657e5 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:40.670754 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1389/2807] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:40 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:40.674887 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782c3cd-d71b-4d04-816f-cefa5eaf965e admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:40.686011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782c3cd-d71b-4d04-816f-cefa5eaf965e admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:40.690793 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b89dd3-0587-4343-9578-123cc1f4b7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:40.691471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b89dd3-0587-4343-9578-123cc1f4b7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:40.691814 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b89dd3-0587-4343-9578-123cc1f4b7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:40.692151 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b89dd3-0587-4343-9578-123cc1f4b7b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:40.692917 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b89dd3-0587-4343-9578-123cc1f4b7b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:40.694573 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b89dd3-0587-4343-9578-123cc1f4b7b5 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:40.705481 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1419/2808] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:40.709572 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d70e1-a8ec-470a-80eb-01edcfe657e5 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:40.721259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d70e1-a8ec-470a-80eb-01edcfe657e5 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:40.726260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d525d-016f-4fb0-8f73-1c41b7a2e3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:40.727758 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d525d-016f-4fb0-8f73-1c41b7a2e3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:40.728075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d525d-016f-4fb0-8f73-1c41b7a2e3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:40.728350 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d525d-016f-4fb0-8f73-1c41b7a2e3a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:40.729050 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d525d-016f-4fb0-8f73-1c41b7a2e3a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:40.730743 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d525d-016f-4fb0-8f73-1c41b7a2e3a5 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:40.741413 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1390/2809] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:40.746265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b89dd3-0587-4343-9578-123cc1f4b7b5 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:40.756377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b89dd3-0587-4343-9578-123cc1f4b7b5 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:40.761331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5913f-62f0-42d7-beea-dcc57fff4679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:40.762067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5913f-62f0-42d7-beea-dcc57fff4679 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:40.762417 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5913f-62f0-42d7-beea-dcc57fff4679 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:40.762797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5913f-62f0-42d7-beea-dcc57fff4679 admin admin] PATH_INFO: `/v3/projects/908cb29b9beb4498a4101797132000fc/users/3f1cbc59f7944d3381efb95b6e607811/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:40.778865 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5913f-62f0-42d7-beea-dcc57fff4679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=908cb29b9beb4498a4101797132000fc, user_id=3f1cbc59f7944d3381efb95b6e607811, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:40.781797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5913f-62f0-42d7-beea-dcc57fff4679 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:40.796116 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1420/2810] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:40 2022] PUT /identity/v3/projects/908cb29b9beb4498a4101797132000fc/users/3f1cbc59f7944d3381efb95b6e607811/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:40.806975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef32c2-acd4-471b-a10a-4e521994e21f None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:40.807260 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef32c2-acd4-471b-a10a-4e521994e21f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:40.807536 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef32c2-acd4-471b-a10a-4e521994e21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:40.837335 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-43ef32c2-acd4-471b-a10a-4e521994e21f None None] MFA Rules not processed for user `3f1cbc59f7944d3381efb95b6e607811`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:40.838891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-43ef32c2-acd4-471b-a10a-4e521994e21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:40.912512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1391/2811] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:40.917753 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5913f-62f0-42d7-beea-dcc57fff4679 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:40.987118 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5913f-62f0-42d7-beea-dcc57fff4679 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:40.991864 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed35516-91e3-4cf4-bd5d-ac548bdcf2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:40.992971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed35516-91e3-4cf4-bd5d-ac548bdcf2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:40.992971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed35516-91e3-4cf4-bd5d-ac548bdcf2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:40.993340 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed35516-91e3-4cf4-bd5d-ac548bdcf2e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:40.993952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed35516-91e3-4cf4-bd5d-ac548bdcf2e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:40.995922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed35516-91e3-4cf4-bd5d-ac548bdcf2e6 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:41.026108 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1421/2812] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:40 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:41.030463 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef32c2-acd4-471b-a10a-4e521994e21f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:41.044778 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef32c2-acd4-471b-a10a-4e521994e21f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:41.104125 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb57224-6b27-4890-956a-3facf8434e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:41.104755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb57224-6b27-4890-956a-3facf8434e5b admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:41.105025 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb57224-6b27-4890-956a-3facf8434e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:41.105297 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb57224-6b27-4890-956a-3facf8434e5b admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:41.105972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb57224-6b27-4890-956a-3facf8434e5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:41.107920 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb57224-6b27-4890-956a-3facf8434e5b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:41.129471 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1392/2813] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:41 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:41.135089 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed35516-91e3-4cf4-bd5d-ac548bdcf2e6 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:41.145651 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed35516-91e3-4cf4-bd5d-ac548bdcf2e6 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:41.151775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecade176-2ec0-4921-ab80-f885d43acc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:41.152402 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecade176-2ec0-4921-ab80-f885d43acc2f admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:41.152677 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecade176-2ec0-4921-ab80-f885d43acc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:41.152943 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecade176-2ec0-4921-ab80-f885d43acc2f admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:41.153656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecade176-2ec0-4921-ab80-f885d43acc2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:41.155296 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecade176-2ec0-4921-ab80-f885d43acc2f admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:41.167820 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1422/2814] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:41.172179 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb57224-6b27-4890-956a-3facf8434e5b admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:41.183626 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb57224-6b27-4890-956a-3facf8434e5b admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:41.188561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62587057-dc96-4f27-90ae-52415debffcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:41.189233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62587057-dc96-4f27-90ae-52415debffcc admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:41.189513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62587057-dc96-4f27-90ae-52415debffcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:41.189915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62587057-dc96-4f27-90ae-52415debffcc admin admin] PATH_INFO: `/v3/projects/99f1024ed38143929379bf649b32c45a/users/d503ad5aa913492f825739169b745202/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:41.205992 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62587057-dc96-4f27-90ae-52415debffcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=99f1024ed38143929379bf649b32c45a, user_id=d503ad5aa913492f825739169b745202, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:41.208688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62587057-dc96-4f27-90ae-52415debffcc admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:41.223086 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1393/2815] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:41 2022] PUT /identity/v3/projects/99f1024ed38143929379bf649b32c45a/users/d503ad5aa913492f825739169b745202/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:41.227003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecade176-2ec0-4921-ab80-f885d43acc2f admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:41.237714 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecade176-2ec0-4921-ab80-f885d43acc2f admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:41.293613 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78a3f4-47e7-4602-a52a-90eb829916f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:41.294246 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78a3f4-47e7-4602-a52a-90eb829916f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:41.294525 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78a3f4-47e7-4602-a52a-90eb829916f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:41.294830 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78a3f4-47e7-4602-a52a-90eb829916f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:41.295675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78a3f4-47e7-4602-a52a-90eb829916f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:41.297476 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78a3f4-47e7-4602-a52a-90eb829916f0 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:41.312462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1423/2816] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:41.313579 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62587057-dc96-4f27-90ae-52415debffcc admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:41.324691 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62587057-dc96-4f27-90ae-52415debffcc admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:41.329614 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26e17f-0348-4cf4-9711-56e6dd81eb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:41.330269 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26e17f-0348-4cf4-9711-56e6dd81eb67 admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:41.330545 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26e17f-0348-4cf4-9711-56e6dd81eb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:41.330847 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26e17f-0348-4cf4-9711-56e6dd81eb67 admin admin] PATH_INFO: `/v3/projects/99f1024ed38143929379bf649b32c45a/users/d503ad5aa913492f825739169b745202/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:41.333213 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26e17f-0348-4cf4-9711-56e6dd81eb67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99f1024ed38143929379bf649b32c45a, user_id=d503ad5aa913492f825739169b745202, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:41.335905 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26e17f-0348-4cf4-9711-56e6dd81eb67 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:41.338349 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:41.349500 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1394/2817] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:41 2022] PUT /identity/v3/projects/99f1024ed38143929379bf649b32c45a/users/d503ad5aa913492f825739169b745202/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:58:41.365898 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f746f-d513-4706-a2b5-7c81065ff7cb None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:41.365898 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f746f-d513-4706-a2b5-7c81065ff7cb None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:41.366331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477f746f-d513-4706-a2b5-7c81065ff7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:41.382744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:41.417378 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-477f746f-d513-4706-a2b5-7c81065ff7cb None None] MFA Rules not processed for user `d503ad5aa913492f825739169b745202`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:41.419784 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-477f746f-d513-4706-a2b5-7c81065ff7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:41.521439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1424/2818] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:41.532429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26e17f-0348-4cf4-9711-56e6dd81eb67 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:41.534354 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9f26e17f-0348-4cf4-9711-56e6dd81eb67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:41.597967 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26e17f-0348-4cf4-9711-56e6dd81eb67 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:41.601593 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d0b63-1b25-497e-a907-260f89daa59a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:41.602216 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d0b63-1b25-497e-a907-260f89daa59a service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:41.602493 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d0b63-1b25-497e-a907-260f89daa59a service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:41.602794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d0b63-1b25-497e-a907-260f89daa59a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:41.662089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d0b63-1b25-497e-a907-260f89daa59a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:41.663608 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d0b63-1b25-497e-a907-260f89daa59a service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:41.674166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1395/2819] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:42.747271 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90959417-589c-4354-8278-cf1a769e3608 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:42.747775 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90959417-589c-4354-8278-cf1a769e3608 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:42.747908 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90959417-589c-4354-8278-cf1a769e3608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:42.766369 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-90959417-589c-4354-8278-cf1a769e3608 None None] MFA Rules not processed for user `2df1b8bc5012478b9ac5b4c2341a8d12`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:42.767585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-90959417-589c-4354-8278-cf1a769e3608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:42.823675 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1425/2820] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:45.996344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d0b63-1b25-497e-a907-260f89daa59a service cinder] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:45.997961 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b28d0b63-1b25-497e-a907-260f89daa59a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:46.010681 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d0b63-1b25-497e-a907-260f89daa59a service cinder] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:46.014541 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acb48f-8ef1-49ed-9d7e-db7ce86417a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:46.015180 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acb48f-8ef1-49ed-9d7e-db7ce86417a0 service cinder] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:46.015549 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acb48f-8ef1-49ed-9d7e-db7ce86417a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:46.015877 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acb48f-8ef1-49ed-9d7e-db7ce86417a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:46.026377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acb48f-8ef1-49ed-9d7e-db7ce86417a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:46.027915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acb48f-8ef1-49ed-9d7e-db7ce86417a0 service cinder] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:46.038078 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1396/2821] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:50.971971 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90959417-589c-4354-8278-cf1a769e3608 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:51.041257 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90959417-589c-4354-8278-cf1a769e3608 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:51.047614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:51.048256 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:51.048532 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:51.048800 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] PATH_INFO: `/v3/users/3f1cbc59f7944d3381efb95b6e607811` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:51.050033 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f1cbc59f7944d3381efb95b6e607811)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:51.051978 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:51.107569 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:51.115355 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f1cbc59f7944d3381efb95b6e607811'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.127829 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f1cbc59f7944d3381efb95b6e607811'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.136643 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f1cbc59f7944d3381efb95b6e607811'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.137411 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f1cbc59f7944d3381efb95b6e607811'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.138092 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f1cbc59f7944d3381efb95b6e607811'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.164358 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1426/2822] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:50 2022] DELETE /identity/v3/users/3f1cbc59f7944d3381efb95b6e607811 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:51.169568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09707245-8cb0-47f4-85b0-03fba607ab93 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:51.170104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09707245-8cb0-47f4-85b0-03fba607ab93 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:51.170104 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09707245-8cb0-47f4-85b0-03fba607ab93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:51.190908 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-09707245-8cb0-47f4-85b0-03fba607ab93 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:51.192096 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-09707245-8cb0-47f4-85b0-03fba607ab93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:51.254598 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1397/2823] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:51.266441 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:51.267877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:51.330855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d8423-93b5-4c83-9ec7-83a7a6ec60e6 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:51.334440 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a36f-eaa9-4d03-b70e-4754a3adce21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:51.335085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a36f-eaa9-4d03-b70e-4754a3adce21 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:51.335356 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a36f-eaa9-4d03-b70e-4754a3adce21 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:51.335619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a36f-eaa9-4d03-b70e-4754a3adce21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:51.345479 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6a36f-eaa9-4d03-b70e-4754a3adce21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:51.346859 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa6a36f-eaa9-4d03-b70e-4754a3adce21 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:51.357183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1427/2824] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:51.742375 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09707245-8cb0-47f4-85b0-03fba607ab93 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:51.743947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-09707245-8cb0-47f4-85b0-03fba607ab93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:51.758595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09707245-8cb0-47f4-85b0-03fba607ab93 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:51.764129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:51.764850 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:51.765189 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:51.765519 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] PATH_INFO: `/v3/projects/908cb29b9beb4498a4101797132000fc` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:51.766781 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=908cb29b9beb4498a4101797132000fc)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:51.769505 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:51.805013 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:INSERT=1,keystone:SELECT=97,keystone:DELETE=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:51.820547 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '908cb29b9beb4498a4101797132000fc', 'request_id': 'req-c4083f71-a342-4665-8c7c-abacbcb5de4b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.827767 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '908cb29b9beb4498a4101797132000fc', 'request_id': 'req-c4083f71-a342-4665-8c7c-abacbcb5de4b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.835367 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '908cb29b9beb4498a4101797132000fc', 'request_id': 'req-c4083f71-a342-4665-8c7c-abacbcb5de4b'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.845944 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] The token cache is being invalidate because project 908cb29b9beb4498a4101797132000fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:51.846229 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.846889 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.848436 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1398/2825] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:51 2022] DELETE /identity/v3/projects/908cb29b9beb4498a4101797132000fc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:51.852600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a36f-eaa9-4d03-b70e-4754a3adce21 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:51.862398 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a36f-eaa9-4d03-b70e-4754a3adce21 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:51.918899 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a44ec-eaf5-47e7-98d1-df73de1c706a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:51.919449 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a44ec-eaf5-47e7-98d1-df73de1c706a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:51.919721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a44ec-eaf5-47e7-98d1-df73de1c706a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:51.919991 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a44ec-eaf5-47e7-98d1-df73de1c706a admin admin] PATH_INFO: `/v3/users/d503ad5aa913492f825739169b745202` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:51.921129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a44ec-eaf5-47e7-98d1-df73de1c706a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d503ad5aa913492f825739169b745202)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:51.923067 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a44ec-eaf5-47e7-98d1-df73de1c706a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:51.970064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bb9a44ec-eaf5-47e7-98d1-df73de1c706a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd503ad5aa913492f825739169b745202'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.980812 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:51.983289 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bb9a44ec-eaf5-47e7-98d1-df73de1c706a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd503ad5aa913492f825739169b745202'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.991391 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bb9a44ec-eaf5-47e7-98d1-df73de1c706a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd503ad5aa913492f825739169b745202'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.992323 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bb9a44ec-eaf5-47e7-98d1-df73de1c706a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd503ad5aa913492f825739169b745202'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:51.992936 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bb9a44ec-eaf5-47e7-98d1-df73de1c706a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd503ad5aa913492f825739169b745202'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:52.005021 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1428/2826] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:51 2022] DELETE /identity/v3/users/d503ad5aa913492f825739169b745202 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:58:52.282437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:52.283953 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:52.300040 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4083f71-a342-4665-8c7c-abacbcb5de4b admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:52.358533 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc01fd-311c-4119-9e4b-7d86f6612672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:52.359204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc01fd-311c-4119-9e4b-7d86f6612672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:52.359483 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc01fd-311c-4119-9e4b-7d86f6612672 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:52.359755 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc01fd-311c-4119-9e4b-7d86f6612672 admin admin] PATH_INFO: `/v3/projects/99f1024ed38143929379bf649b32c45a` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:52.360938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc01fd-311c-4119-9e4b-7d86f6612672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99f1024ed38143929379bf649b32c45a)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:52.362870 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc01fd-311c-4119-9e4b-7d86f6612672 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:52.409955 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f6cc01fd-311c-4119-9e4b-7d86f6612672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99f1024ed38143929379bf649b32c45a', 'request_id': 'req-f6cc01fd-311c-4119-9e4b-7d86f6612672'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:52.416355 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f6cc01fd-311c-4119-9e4b-7d86f6612672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99f1024ed38143929379bf649b32c45a', 'request_id': 'req-f6cc01fd-311c-4119-9e4b-7d86f6612672'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:52.423929 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f6cc01fd-311c-4119-9e4b-7d86f6612672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99f1024ed38143929379bf649b32c45a', 'request_id': 'req-f6cc01fd-311c-4119-9e4b-7d86f6612672'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:52.435622 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f6cc01fd-311c-4119-9e4b-7d86f6612672 admin admin] The token cache is being invalidate because project 99f1024ed38143929379bf649b32c45a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:52.435903 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f6cc01fd-311c-4119-9e4b-7d86f6612672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:52.436527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-f6cc01fd-311c-4119-9e4b-7d86f6612672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:52.438048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1399/2827] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:52 2022] DELETE /identity/v3/projects/99f1024ed38143929379bf649b32c45a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:52.444279 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86bb39-8a8f-4666-ad58-2879a3568d06 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:52.444554 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86bb39-8a8f-4666-ad58-2879a3568d06 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:52.444830 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86bb39-8a8f-4666-ad58-2879a3568d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:52.466671 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-4f86bb39-8a8f-4666-ad58-2879a3568d06 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:52.467779 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4f86bb39-8a8f-4666-ad58-2879a3568d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:52.528614 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1429/2828] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:52.537362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad64f01-2211-4f02-92ce-e8c6f92e4e09 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:52.537650 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad64f01-2211-4f02-92ce-e8c6f92e4e09 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:52.537960 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad64f01-2211-4f02-92ce-e8c6f92e4e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:52.559164 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-4ad64f01-2211-4f02-92ce-e8c6f92e4e09 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:52.560416 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-4ad64f01-2211-4f02-92ce-e8c6f92e4e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:52.568486 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1400/2829] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:52.573831 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86bb39-8a8f-4666-ad58-2879a3568d06 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:52.583683 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86bb39-8a8f-4666-ad58-2879a3568d06 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:52.588424 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821ef62-55c3-4974-b64d-e7e29523a720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:52.589025 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821ef62-55c3-4974-b64d-e7e29523a720 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:52.589297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821ef62-55c3-4974-b64d-e7e29523a720 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:52.589563 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821ef62-55c3-4974-b64d-e7e29523a720 admin admin] PATH_INFO: `/v3/domains` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:52.590245 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821ef62-55c3-4974-b64d-e7e29523a720 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87004) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:58:52.590544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821ef62-55c3-4974-b64d-e7e29523a720 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:52.592159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821ef62-55c3-4974-b64d-e7e29523a720 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:52.603619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1430/2830] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:58:52.609267 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad64f01-2211-4f02-92ce-e8c6f92e4e09 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:52.619195 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad64f01-2211-4f02-92ce-e8c6f92e4e09 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:52.624063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c256c25-b262-4fa0-9ebc-d209e7209735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:52.624665 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c256c25-b262-4fa0-9ebc-d209e7209735 admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:52.624942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c256c25-b262-4fa0-9ebc-d209e7209735 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:52.625211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c256c25-b262-4fa0-9ebc-d209e7209735 admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:52.625863 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c256c25-b262-4fa0-9ebc-d209e7209735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:52.627773 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c256c25-b262-4fa0-9ebc-d209e7209735 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:52.654971 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1401/2831] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:52 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:52.659259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821ef62-55c3-4974-b64d-e7e29523a720 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:52.669712 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821ef62-55c3-4974-b64d-e7e29523a720 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:52.725829 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524db69-e1b5-4438-b88a-b32b06efed36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:52.726352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524db69-e1b5-4438-b88a-b32b06efed36 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:52.726644 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524db69-e1b5-4438-b88a-b32b06efed36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:52.726915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524db69-e1b5-4438-b88a-b32b06efed36 admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:52.727567 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524db69-e1b5-4438-b88a-b32b06efed36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:52.729690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524db69-e1b5-4438-b88a-b32b06efed36 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:52.751749 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1431/2832] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:52 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:52.755801 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c256c25-b262-4fa0-9ebc-d209e7209735 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:52.767790 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c256c25-b262-4fa0-9ebc-d209e7209735 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:52.772576 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ce8aa-d946-4896-97ea-f304bdcf096a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:52.773206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ce8aa-d946-4896-97ea-f304bdcf096a admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:52.773510 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ce8aa-d946-4896-97ea-f304bdcf096a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:52.773806 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ce8aa-d946-4896-97ea-f304bdcf096a admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:52.774511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ce8aa-d946-4896-97ea-f304bdcf096a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:52.776232 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ce8aa-d946-4896-97ea-f304bdcf096a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:52.789582 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1402/2833] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:52.791567 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524db69-e1b5-4438-b88a-b32b06efed36 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:52.803238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524db69-e1b5-4438-b88a-b32b06efed36 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:52.811562 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94734983-c955-466f-9a95-5d94e12aa4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:52.812273 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94734983-c955-466f-9a95-5d94e12aa4fe admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:52.812620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94734983-c955-466f-9a95-5d94e12aa4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:52.812965 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94734983-c955-466f-9a95-5d94e12aa4fe admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:52.813733 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94734983-c955-466f-9a95-5d94e12aa4fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:52.815431 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94734983-c955-466f-9a95-5d94e12aa4fe admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:52.827787 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1432/2834] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:52.831595 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ce8aa-d946-4896-97ea-f304bdcf096a admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:52.842089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ce8aa-d946-4896-97ea-f304bdcf096a admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:52.846962 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5078ced-ce77-4035-b05c-1731f8a026ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:52.847800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5078ced-ce77-4035-b05c-1731f8a026ee admin admin] REQUEST_METHOD: `PUT` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:52.848080 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5078ced-ce77-4035-b05c-1731f8a026ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:52.848357 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5078ced-ce77-4035-b05c-1731f8a026ee admin admin] PATH_INFO: `/v3/projects/815e2bcbdd2644968029839efc95174e/users/74e60f6e3c01460cbcc968a0758e30a6/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:52.851995 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:52.866844 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5078ced-ce77-4035-b05c-1731f8a026ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=815e2bcbdd2644968029839efc95174e, user_id=74e60f6e3c01460cbcc968a0758e30a6, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:52.871603 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5078ced-ce77-4035-b05c-1731f8a026ee admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:52.893397 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1403/2835] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:52 2022] PUT /identity/v3/projects/815e2bcbdd2644968029839efc95174e/users/74e60f6e3c01460cbcc968a0758e30a6/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:52.899179 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b66ba-77bc-4967-aa46-aceed5c70773 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:52.899786 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b66ba-77bc-4967-aa46-aceed5c70773 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:52.900049 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b66ba-77bc-4967-aa46-aceed5c70773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:52.934690 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-dc6b66ba-77bc-4967-aa46-aceed5c70773 None None] MFA Rules not processed for user `74e60f6e3c01460cbcc968a0758e30a6`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:52.936226 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-dc6b66ba-77bc-4967-aa46-aceed5c70773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:52.991540 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:53.028868 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1433/2836] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:53.033512 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5078ced-ce77-4035-b05c-1731f8a026ee admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:53.097548 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5078ced-ce77-4035-b05c-1731f8a026ee admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:53.102515 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6717f1-4892-4c6a-bf67-b68f76b6b8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:53.103708 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6717f1-4892-4c6a-bf67-b68f76b6b8de admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:53.104087 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6717f1-4892-4c6a-bf67-b68f76b6b8de admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:53.104365 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6717f1-4892-4c6a-bf67-b68f76b6b8de admin admin] PATH_INFO: `/v3/projects` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:53.105022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6717f1-4892-4c6a-bf67-b68f76b6b8de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:53.107012 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6717f1-4892-4c6a-bf67-b68f76b6b8de admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:53.133983 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1404/2837] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:58:53 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:53.138406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b66ba-77bc-4967-aa46-aceed5c70773 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:53.149064 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b66ba-77bc-4967-aa46-aceed5c70773 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:53.205689 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76991e4b-1395-435b-b5a2-5a292c5779fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:53.206307 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76991e4b-1395-435b-b5a2-5a292c5779fe admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:53.206585 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76991e4b-1395-435b-b5a2-5a292c5779fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:53.206889 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76991e4b-1395-435b-b5a2-5a292c5779fe admin admin] PATH_INFO: `/v3/users` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:53.207544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76991e4b-1395-435b-b5a2-5a292c5779fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:53.209458 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76991e4b-1395-435b-b5a2-5a292c5779fe admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:53.231693 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1434/2838] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:58:53 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:58:53.235924 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6717f1-4892-4c6a-bf67-b68f76b6b8de admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:53.247344 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6717f1-4892-4c6a-bf67-b68f76b6b8de admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:53.251960 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d4c1a-6c6b-423d-b01c-89ef302b7a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:53.252602 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d4c1a-6c6b-423d-b01c-89ef302b7a3a admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:53.252835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d4c1a-6c6b-423d-b01c-89ef302b7a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:53.253101 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d4c1a-6c6b-423d-b01c-89ef302b7a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:53.253788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d4c1a-6c6b-423d-b01c-89ef302b7a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:53.255437 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d4c1a-6c6b-423d-b01c-89ef302b7a3a admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:53.266171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1405/2839] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:53.270387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76991e4b-1395-435b-b5a2-5a292c5779fe admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:53.281865 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76991e4b-1395-435b-b5a2-5a292c5779fe admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:53.286671 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcd453-b65f-4c02-a3a3-2825cda09766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:53.287321 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcd453-b65f-4c02-a3a3-2825cda09766 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:53.287619 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcd453-b65f-4c02-a3a3-2825cda09766 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:53.287903 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcd453-b65f-4c02-a3a3-2825cda09766 admin admin] PATH_INFO: `/v3/projects/1f7c9d2c9cac47e08b8d991633ec1c09/users/5b5281adf0bc43eb98e0a415bdd8ba14/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:53.303948 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fcd453-b65f-4c02-a3a3-2825cda09766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7c9d2c9cac47e08b8d991633ec1c09, user_id=5b5281adf0bc43eb98e0a415bdd8ba14, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:53.307095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fcd453-b65f-4c02-a3a3-2825cda09766 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:53.320829 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1435/2840] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:53 2022] PUT /identity/v3/projects/1f7c9d2c9cac47e08b8d991633ec1c09/users/5b5281adf0bc43eb98e0a415bdd8ba14/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:53.325022 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d4c1a-6c6b-423d-b01c-89ef302b7a3a admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:53.335224 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d4c1a-6c6b-423d-b01c-89ef302b7a3a admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:53.393233 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb1a40-eafe-4faa-8013-8e4e5cdc9c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:53.393838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb1a40-eafe-4faa-8013-8e4e5cdc9c49 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:53.394144 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb1a40-eafe-4faa-8013-8e4e5cdc9c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:53.394414 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb1a40-eafe-4faa-8013-8e4e5cdc9c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:53.395136 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb1a40-eafe-4faa-8013-8e4e5cdc9c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:53.396713 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb1a40-eafe-4faa-8013-8e4e5cdc9c49 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:53.406492 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1406/2841] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:58:53.410768 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcd453-b65f-4c02-a3a3-2825cda09766 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:53.421304 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcd453-b65f-4c02-a3a3-2825cda09766 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:53.425952 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58650035-7ba3-4541-a4a9-790695477d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:53.426604 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58650035-7ba3-4541-a4a9-790695477d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:53.426918 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58650035-7ba3-4541-a4a9-790695477d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:53.427193 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58650035-7ba3-4541-a4a9-790695477d6b admin admin] PATH_INFO: `/v3/projects/1f7c9d2c9cac47e08b8d991633ec1c09/users/5b5281adf0bc43eb98e0a415bdd8ba14/roles/93eb204498fc49d297bcd1841e00818d` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:53.429244 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58650035-7ba3-4541-a4a9-790695477d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7c9d2c9cac47e08b8d991633ec1c09, user_id=5b5281adf0bc43eb98e0a415bdd8ba14, role_id=93eb204498fc49d297bcd1841e00818d)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:53.432834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58650035-7ba3-4541-a4a9-790695477d6b admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:53.443176 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1436/2842] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:58:53 2022] PUT /identity/v3/projects/1f7c9d2c9cac47e08b8d991633ec1c09/users/5b5281adf0bc43eb98e0a415bdd8ba14/roles/93eb204498fc49d297bcd1841e00818d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:53.455255 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d7cc3-a369-4ebb-bd8f-9cabbedae28a None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:53.455616 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d7cc3-a369-4ebb-bd8f-9cabbedae28a None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:53.455958 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d7cc3-a369-4ebb-bd8f-9cabbedae28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:53.483970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-f33d7cc3-a369-4ebb-bd8f-9cabbedae28a None None] MFA Rules not processed for user `5b5281adf0bc43eb98e0a415bdd8ba14`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:53.485177 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-f33d7cc3-a369-4ebb-bd8f-9cabbedae28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:53.529454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:53.554498 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1407/2843] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:58:53.566637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58650035-7ba3-4541-a4a9-790695477d6b admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:53.568368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-58650035-7ba3-4541-a4a9-790695477d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:53.627656 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58650035-7ba3-4541-a4a9-790695477d6b admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:53.631364 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5915e-d128-44c6-be62-7601e255403f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:53.631980 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5915e-d128-44c6-be62-7601e255403f service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:53.632259 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5915e-d128-44c6-be62-7601e255403f service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:53.632530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5915e-d128-44c6-be62-7601e255403f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:53.673406 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:53.693396 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5915e-d128-44c6-be62-7601e255403f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:53.694856 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5915e-d128-44c6-be62-7601e255403f service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:53.704482 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1437/2844] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:56.277524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d7cc3-a369-4ebb-bd8f-9cabbedae28a None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:56.342975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d7cc3-a369-4ebb-bd8f-9cabbedae28a None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:56.347821 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:56.348491 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:56.348884 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:56.349163 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] PATH_INFO: `/v3/users/74e60f6e3c01460cbcc968a0758e30a6` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:56.350451 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=74e60f6e3c01460cbcc968a0758e30a6)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:56.352405 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:56.402722 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74e60f6e3c01460cbcc968a0758e30a6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:56.410000 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74e60f6e3c01460cbcc968a0758e30a6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:56.410780 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74e60f6e3c01460cbcc968a0758e30a6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:56.424033 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74e60f6e3c01460cbcc968a0758e30a6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:56.432794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74e60f6e3c01460cbcc968a0758e30a6'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:56.439063 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1408/2845] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:56 2022] DELETE /identity/v3/users/74e60f6e3c01460cbcc968a0758e30a6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:56.443629 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738383d-3588-4047-a12b-9f39a78c246d None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:56.444174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738383d-3588-4047-a12b-9f39a78c246d None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:56.444174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738383d-3588-4047-a12b-9f39a78c246d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:56.465223 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-6738383d-3588-4047-a12b-9f39a78c246d None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:58:56.466439 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6738383d-3588-4047-a12b-9f39a78c246d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:56.537100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1438/2846] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:58:56.547723 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:56.549149 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:56.609897 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c7b51-bab1-4938-84ec-dd4a27cd46ac admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:56.613549 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40d4d2-009e-4436-87be-42c3b34689b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:56.614161 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40d4d2-009e-4436-87be-42c3b34689b6 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:56.614440 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40d4d2-009e-4436-87be-42c3b34689b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:56.614741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40d4d2-009e-4436-87be-42c3b34689b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:56.625170 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40d4d2-009e-4436-87be-42c3b34689b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:56.626833 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40d4d2-009e-4436-87be-42c3b34689b6 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:56.638938 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1409/2847] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:58:56.932891 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738383d-3588-4047-a12b-9f39a78c246d None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:56.934368 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6738383d-3588-4047-a12b-9f39a78c246d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:56.946312 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738383d-3588-4047-a12b-9f39a78c246d None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:56.951778 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:56.952428 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:56.952732 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:56.953003 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] PATH_INFO: `/v3/projects/815e2bcbdd2644968029839efc95174e` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:56.954430 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=815e2bcbdd2644968029839efc95174e)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:56.956383 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:57.003753 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '815e2bcbdd2644968029839efc95174e', 'request_id': 'req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.010786 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '815e2bcbdd2644968029839efc95174e', 'request_id': 'req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.017844 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '815e2bcbdd2644968029839efc95174e', 'request_id': 'req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.029190 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] The token cache is being invalidate because project 815e2bcbdd2644968029839efc95174e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:57.029475 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.030111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.031662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1439/2848] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:56 2022] DELETE /identity/v3/projects/815e2bcbdd2644968029839efc95174e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:58:57.035794 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40d4d2-009e-4436-87be-42c3b34689b6 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:57.040651 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:57.046653 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40d4d2-009e-4436-87be-42c3b34689b6 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:57.105158 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb71fbd-61e7-4d45-9d43-d0bf9d660efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:57.105712 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb71fbd-61e7-4d45-9d43-d0bf9d660efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:57.105996 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb71fbd-61e7-4d45-9d43-d0bf9d660efc admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:57.106271 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb71fbd-61e7-4d45-9d43-d0bf9d660efc admin admin] PATH_INFO: `/v3/users/5b5281adf0bc43eb98e0a415bdd8ba14` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:57.107501 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb71fbd-61e7-4d45-9d43-d0bf9d660efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b5281adf0bc43eb98e0a415bdd8ba14)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:57.109400 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb71fbd-61e7-4d45-9d43-d0bf9d660efc admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:57.165845 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-beb71fbd-61e7-4d45-9d43-d0bf9d660efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b5281adf0bc43eb98e0a415bdd8ba14'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.175187 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-beb71fbd-61e7-4d45-9d43-d0bf9d660efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b5281adf0bc43eb98e0a415bdd8ba14'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.176002 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-beb71fbd-61e7-4d45-9d43-d0bf9d660efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b5281adf0bc43eb98e0a415bdd8ba14'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.193183 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-beb71fbd-61e7-4d45-9d43-d0bf9d660efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b5281adf0bc43eb98e0a415bdd8ba14'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.203343 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-beb71fbd-61e7-4d45-9d43-d0bf9d660efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b5281adf0bc43eb98e0a415bdd8ba14'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.209864 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1410/2849] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:58:57 2022] DELETE /identity/v3/users/5b5281adf0bc43eb98e0a415bdd8ba14 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:58:57.492362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:58:57.495303 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:58:57.518009 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce543d-b1a7-4e23-b6ef-c777fca3c4e0 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:58:57.574222 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:58:57.574922 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:58:57.575200 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:58:57.575471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] PATH_INFO: `/v3/projects/1f7c9d2c9cac47e08b8d991633ec1c09` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:58:57.576620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f7c9d2c9cac47e08b8d991633ec1c09)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:58:57.578512 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:58:57.584668 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:58:57.626617 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f7c9d2c9cac47e08b8d991633ec1c09', 'request_id': 'req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.632658 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f7c9d2c9cac47e08b8d991633ec1c09', 'request_id': 'req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.640748 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f7c9d2c9cac47e08b8d991633ec1c09', 'request_id': 'req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.651426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] The token cache is being invalidate because project 1f7c9d2c9cac47e08b8d991633ec1c09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:58:57.651701 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.652335 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:58:57.653843 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1440/2850] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:58:57 2022] DELETE /identity/v3/projects/1f7c9d2c9cac47e08b8d991633ec1c09 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:59:02.918361 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6206d1-3da3-488c-9ace-e21e656122da None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:02.919048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6206d1-3da3-488c-9ace-e21e656122da None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:02.919048 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6206d1-3da3-488c-9ace-e21e656122da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:02.940586 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-6d6206d1-3da3-488c-9ace-e21e656122da None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:59:02.942340 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-6d6206d1-3da3-488c-9ace-e21e656122da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:03.007688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1411/2851] 10.222.0.86 () {58 vars in 1009 bytes} [Fri Aug 5 19:59:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:59:03.020104 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:03.021521 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:03.098919 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060a3dd-5e10-40b3-ba0e-970a7ecbd133 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:03.104132 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e26754-f744-4060-8851-122172e58b18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:03.104691 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e26754-f744-4060-8851-122172e58b18 service swift] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:03.104967 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e26754-f744-4060-8851-122172e58b18 service swift] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:03.105238 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e26754-f744-4060-8851-122172e58b18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:03.115424 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e26754-f744-4060-8851-122172e58b18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:03.117053 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e26754-f744-4060-8851-122172e58b18 service swift] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:03.127011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1441/2852] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:59:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:59:13.000189 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=77,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:59:13.123637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:59:17.564295 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97d71c-6ffb-4422-aa0d-95923e155bc8 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:17.564795 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97d71c-6ffb-4422-aa0d-95923e155bc8 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:17.564892 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97d71c-6ffb-4422-aa0d-95923e155bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:17.583693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-ef97d71c-6ffb-4422-aa0d-95923e155bc8 None None] MFA Rules not processed for user `7fc4868f25364311a504224fcd2c5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:59:17.584887 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ef97d71c-6ffb-4422-aa0d-95923e155bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:17.592909 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1412/2853] 10.222.0.86 () {58 vars in 967 bytes} [Fri Aug 5 19:59:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:59:17.605929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e26754-f744-4060-8851-122172e58b18 service swift] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:17.616469 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e26754-f744-4060-8851-122172e58b18 service swift] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:17.621994 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ef412-c774-49bb-8025-45d288529534 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8adb0fd1fbc4268b70f3936036e550b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:17.622600 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ef412-c774-49bb-8025-45d288529534 service swift] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:17.622915 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ef412-c774-49bb-8025-45d288529534 service swift] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:17.623191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ef412-c774-49bb-8025-45d288529534 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:17.631496 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ef412-c774-49bb-8025-45d288529534 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:17.633099 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ef412-c774-49bb-8025-45d288529534 service swift] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:17.641742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1442/2854] 10.222.0.86 () {60 vars in 1384 bytes} [Fri Aug 5 19:59:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 05 19:59:19.546242 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97d71c-6ffb-4422-aa0d-95923e155bc8 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:19.547703 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ef97d71c-6ffb-4422-aa0d-95923e155bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:19.616640 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97d71c-6ffb-4422-aa0d-95923e155bc8 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:19.622142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d069ff-47c5-407a-a108-afea98160947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:19.622899 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d069ff-47c5-407a-a108-afea98160947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:19.623174 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d069ff-47c5-407a-a108-afea98160947 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:19.623439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d069ff-47c5-407a-a108-afea98160947 admin admin] PATH_INFO: `/v3/users/2df1b8bc5012478b9ac5b4c2341a8d12` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:19.624604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d069ff-47c5-407a-a108-afea98160947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2df1b8bc5012478b9ac5b4c2341a8d12)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:19.626612 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d069ff-47c5-407a-a108-afea98160947 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:19.673517 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-31d069ff-47c5-407a-a108-afea98160947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2df1b8bc5012478b9ac5b4c2341a8d12'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:19.680277 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-31d069ff-47c5-407a-a108-afea98160947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2df1b8bc5012478b9ac5b4c2341a8d12'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:19.680965 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-31d069ff-47c5-407a-a108-afea98160947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2df1b8bc5012478b9ac5b4c2341a8d12'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:19.695379 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-31d069ff-47c5-407a-a108-afea98160947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2df1b8bc5012478b9ac5b4c2341a8d12'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:19.704421 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-31d069ff-47c5-407a-a108-afea98160947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2df1b8bc5012478b9ac5b4c2341a8d12'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:19.710389 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1413/2855] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:59:19 2022] DELETE /identity/v3/users/2df1b8bc5012478b9ac5b4c2341a8d12 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:59:19.883042 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ef412-c774-49bb-8025-45d288529534 service swift] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:19.895354 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ef412-c774-49bb-8025-45d288529534 service swift] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:19.956111 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb30996-91a9-41d8-880e-4f14d4a37f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:19.956744 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb30996-91a9-41d8-880e-4f14d4a37f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:19.957022 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb30996-91a9-41d8-880e-4f14d4a37f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:19.957294 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb30996-91a9-41d8-880e-4f14d4a37f5a admin admin] PATH_INFO: `/v3/projects/1a0f6dc247f547a79f8908d072bc49e3` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:19.958434 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb30996-91a9-41d8-880e-4f14d4a37f5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a0f6dc247f547a79f8908d072bc49e3)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:19.960375 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb30996-91a9-41d8-880e-4f14d4a37f5a admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:20.006288 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9bb30996-91a9-41d8-880e-4f14d4a37f5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a0f6dc247f547a79f8908d072bc49e3', 'request_id': 'req-9bb30996-91a9-41d8-880e-4f14d4a37f5a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:20.013081 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9bb30996-91a9-41d8-880e-4f14d4a37f5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a0f6dc247f547a79f8908d072bc49e3', 'request_id': 'req-9bb30996-91a9-41d8-880e-4f14d4a37f5a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:20.021360 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9bb30996-91a9-41d8-880e-4f14d4a37f5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a0f6dc247f547a79f8908d072bc49e3', 'request_id': 'req-9bb30996-91a9-41d8-880e-4f14d4a37f5a'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:20.032863 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9bb30996-91a9-41d8-880e-4f14d4a37f5a admin admin] The token cache is being invalidate because project 1a0f6dc247f547a79f8908d072bc49e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:59:20.033143 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9bb30996-91a9-41d8-880e-4f14d4a37f5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:20.033766 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-9bb30996-91a9-41d8-880e-4f14d4a37f5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:20.035308 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1443/2856] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:59:19 2022] DELETE /identity/v3/projects/1a0f6dc247f547a79f8908d072bc49e3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:59:20.040188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cfbb4-7f8b-4cab-bc6f-585254e3d93e None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:20.040477 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cfbb4-7f8b-4cab-bc6f-585254e3d93e None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:20.040788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cfbb4-7f8b-4cab-bc6f-585254e3d93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:20.061287 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-558cfbb4-7f8b-4cab-bc6f-585254e3d93e None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:59:20.062513 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-558cfbb4-7f8b-4cab-bc6f-585254e3d93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:20.122307 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1414/2857] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 05 19:59:20.130007 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a1e15-3cdf-4d3c-9a7b-046275ff1351 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:20.130224 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a1e15-3cdf-4d3c-9a7b-046275ff1351 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:20.130495 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a1e15-3cdf-4d3c-9a7b-046275ff1351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:20.151801 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-ce1a1e15-3cdf-4d3c-9a7b-046275ff1351 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:59:20.153086 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-ce1a1e15-3cdf-4d3c-9a7b-046275ff1351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:20.160890 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1444/2858] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:59:20.164989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cfbb4-7f8b-4cab-bc6f-585254e3d93e None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:20.174923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cfbb4-7f8b-4cab-bc6f-585254e3d93e None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:20.179768 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff3812-0163-486c-a850-dfd07fa107b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:20.180392 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff3812-0163-486c-a850-dfd07fa107b9 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:20.180670 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff3812-0163-486c-a850-dfd07fa107b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:20.180942 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff3812-0163-486c-a850-dfd07fa107b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:20.181903 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff3812-0163-486c-a850-dfd07fa107b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87005) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:59:20.182214 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff3812-0163-486c-a850-dfd07fa107b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:20.184188 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff3812-0163-486c-a850-dfd07fa107b9 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:20.195484 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:59:20.203642 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1415/2859] 10.222.0.86 () {58 vars in 1153 bytes} [Fri Aug 5 19:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:59:20.204782 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a1e15-3cdf-4d3c-9a7b-046275ff1351 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:20.214739 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a1e15-3cdf-4d3c-9a7b-046275ff1351 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:20.219386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c5f93-478c-4130-9e86-fb6d1f884c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:20.219989 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c5f93-478c-4130-9e86-fb6d1f884c81 admin admin] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:20.220266 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c5f93-478c-4130-9e86-fb6d1f884c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:20.220537 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c5f93-478c-4130-9e86-fb6d1f884c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:20.221192 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c5f93-478c-4130-9e86-fb6d1f884c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:20.223110 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c5f93-478c-4130-9e86-fb6d1f884c81 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:20.252290 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1445/2860] 10.222.0.86 () {60 vars in 1153 bytes} [Fri Aug 5 19:59:20 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:59:20.256568 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff3812-0163-486c-a850-dfd07fa107b9 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:20.267531 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff3812-0163-486c-a850-dfd07fa107b9 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:20.325972 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3cdd2-f064-4626-83e2-d1cc75068bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:20.326587 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3cdd2-f064-4626-83e2-d1cc75068bde admin admin] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:20.326895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3cdd2-f064-4626-83e2-d1cc75068bde admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:20.327186 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3cdd2-f064-4626-83e2-d1cc75068bde admin admin] PATH_INFO: `/v3/users` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:20.327834 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3cdd2-f064-4626-83e2-d1cc75068bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:20.329733 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3cdd2-f064-4626-83e2-d1cc75068bde admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:20.352275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1416/2861] 10.222.0.86 () {60 vars in 1144 bytes} [Fri Aug 5 19:59:20 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:59:20.356558 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c5f93-478c-4130-9e86-fb6d1f884c81 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:20.366918 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c5f93-478c-4130-9e86-fb6d1f884c81 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:20.372745 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a28c03-0dc7-4d0a-94fa-89d0edab8230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:20.373386 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a28c03-0dc7-4d0a-94fa-89d0edab8230 admin admin] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:20.373713 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a28c03-0dc7-4d0a-94fa-89d0edab8230 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:20.374085 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a28c03-0dc7-4d0a-94fa-89d0edab8230 admin admin] PATH_INFO: `/v3/roles` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:20.374817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a28c03-0dc7-4d0a-94fa-89d0edab8230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:20.376494 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a28c03-0dc7-4d0a-94fa-89d0edab8230 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:20.388616 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1446/2862] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:59:20.394895 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3cdd2-f064-4626-83e2-d1cc75068bde admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:20.408082 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3cdd2-f064-4626-83e2-d1cc75068bde admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:20.412915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e86b6-6185-4e76-99ea-01713e31fce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:20.413523 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e86b6-6185-4e76-99ea-01713e31fce2 admin admin] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:20.413800 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e86b6-6185-4e76-99ea-01713e31fce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:20.414068 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e86b6-6185-4e76-99ea-01713e31fce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:20.414796 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e86b6-6185-4e76-99ea-01713e31fce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:20.416669 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e86b6-6185-4e76-99ea-01713e31fce2 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:20.427244 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1417/2863] 10.222.0.86 () {58 vars in 1122 bytes} [Fri Aug 5 19:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:59:20.431939 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a28c03-0dc7-4d0a-94fa-89d0edab8230 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:20.442410 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a28c03-0dc7-4d0a-94fa-89d0edab8230 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:20.447245 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739c623-afa3-4455-9659-49456d1c5318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:20.448450 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739c623-afa3-4455-9659-49456d1c5318 admin admin] REQUEST_METHOD: `PUT` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:20.448752 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739c623-afa3-4455-9659-49456d1c5318 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:20.449050 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739c623-afa3-4455-9659-49456d1c5318 admin admin] PATH_INFO: `/v3/projects/f26f664bd488421d91ccb9f8bbbc8335/users/1a1068c6df024ba9a2bfacba1d0c014c/roles/fe6c7625134a449c8395ab84ffd82371` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:20.464624 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739c623-afa3-4455-9659-49456d1c5318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26f664bd488421d91ccb9f8bbbc8335, user_id=1a1068c6df024ba9a2bfacba1d0c014c, role_id=fe6c7625134a449c8395ab84ffd82371)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:20.467328 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739c623-afa3-4455-9659-49456d1c5318 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:20.480512 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1447/2864] 10.222.0.86 () {60 vars in 1483 bytes} [Fri Aug 5 19:59:20 2022] PUT /identity/v3/projects/f26f664bd488421d91ccb9f8bbbc8335/users/1a1068c6df024ba9a2bfacba1d0c014c/roles/fe6c7625134a449c8395ab84ffd82371 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:59:20.491275 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c4563-1e65-49fa-a677-8bdbbe1acc0f None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:20.491560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c4563-1e65-49fa-a677-8bdbbe1acc0f None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:20.491835 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c4563-1e65-49fa-a677-8bdbbe1acc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:20.522115 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-e51c4563-1e65-49fa-a677-8bdbbe1acc0f None None] MFA Rules not processed for user `1a1068c6df024ba9a2bfacba1d0c014c`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:59:20.523353 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e51c4563-1e65-49fa-a677-8bdbbe1acc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:20.587797 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1418/2865] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:59:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:59:20.598802 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739c623-afa3-4455-9659-49456d1c5318 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:20.600507 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d739c623-afa3-4455-9659-49456d1c5318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:20.618816 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:59:20.664462 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739c623-afa3-4455-9659-49456d1c5318 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:20.669929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0617ac-15ba-4adb-98df-cfe1f7df9111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afea7d279292424490beff0481ff1512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:20.670538 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0617ac-15ba-4adb-98df-cfe1f7df9111 service cinder] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:20.670860 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0617ac-15ba-4adb-98df-cfe1f7df9111 service cinder] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:20.671129 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0617ac-15ba-4adb-98df-cfe1f7df9111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:20.679603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0617ac-15ba-4adb-98df-cfe1f7df9111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:20.681016 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0617ac-15ba-4adb-98df-cfe1f7df9111 service cinder] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:20.691154 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1448/2866] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:59:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:59:20.717221 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c4563-1e65-49fa-a677-8bdbbe1acc0f None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:20.778143 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c4563-1e65-49fa-a677-8bdbbe1acc0f None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:20.782736 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:20.783363 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:20.783641 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:20.783914 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] PATH_INFO: `/v3/users/1a1068c6df024ba9a2bfacba1d0c014c` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:20.785230 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a1068c6df024ba9a2bfacba1d0c014c)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:20.787164 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:20.798454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:59:20.833656 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a1068c6df024ba9a2bfacba1d0c014c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:20.840849 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a1068c6df024ba9a2bfacba1d0c014c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:20.841530 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a1068c6df024ba9a2bfacba1d0c014c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:20.853193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a1068c6df024ba9a2bfacba1d0c014c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:20.861072 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a1068c6df024ba9a2bfacba1d0c014c'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:20.867129 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1419/2867] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 19:59:20 2022] DELETE /identity/v3/users/1a1068c6df024ba9a2bfacba1d0c014c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:59:20.872387 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5241e2b-a370-4921-b67f-df855db59459 None None] REQUEST_METHOD: `POST` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:20.872673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5241e2b-a370-4921-b67f-df855db59459 None None] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:20.872975 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5241e2b-a370-4921-b67f-df855db59459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:20.892174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.auth.core [None req-c5241e2b-a370-4921-b67f-df855db59459 None None] MFA Rules not processed for user `047b4b41672048d8889a2b3bb55b2d97`. Rule list: `[]` (Enabled: `True`). {{(pid=87004) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:59:20.893377 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c5241e2b-a370-4921-b67f-df855db59459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:20.952209 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1449/2868] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 19:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 19:59:20.961823 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:20.963293 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:21.028304 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac79d98-d445-4a73-ad28-c1d2a0938dc2 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:21.035557 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4271a-52e7-4070-9465-9c8cf02f02d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:21.036166 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4271a-52e7-4070-9465-9c8cf02f02d7 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:21.036439 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4271a-52e7-4070-9465-9c8cf02f02d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:21.036704 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4271a-52e7-4070-9465-9c8cf02f02d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:21.047478 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4271a-52e7-4070-9465-9c8cf02f02d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:21.048891 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4271a-52e7-4070-9465-9c8cf02f02d7 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:21.059524 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1420/2869] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 19:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 19:59:21.329990 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5241e2b-a370-4921-b67f-df855db59459 None None] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:59:21.331451 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-c5241e2b-a370-4921-b67f-df855db59459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:59:21.346156 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5241e2b-a370-4921-b67f-df855db59459 None None] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:59:21.351662 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:59:21.352367 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:59:21.352672 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:59:21.352949 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] PATH_INFO: `/v3/projects/f26f664bd488421d91ccb9f8bbbc8335` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:59:21.355471 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f26f664bd488421d91ccb9f8bbbc8335)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:59:21.358705 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:59:21.409444 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f26f664bd488421d91ccb9f8bbbc8335', 'request_id': 'req-d3fcd7c4-98ea-42f3-8c85-0932345a3384'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:21.415635 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f26f664bd488421d91ccb9f8bbbc8335', 'request_id': 'req-d3fcd7c4-98ea-42f3-8c85-0932345a3384'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:21.423520 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f26f664bd488421d91ccb9f8bbbc8335', 'request_id': 'req-d3fcd7c4-98ea-42f3-8c85-0932345a3384'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:21.435070 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] The token cache is being invalidate because project f26f664bd488421d91ccb9f8bbbc8335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 19:59:21.435362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:21.436010 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 19:59:21.437528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1450/2870] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 19:59:21 2022] DELETE /identity/v3/projects/f26f664bd488421d91ccb9f8bbbc8335 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:59:31.054273 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:DELETE=10,keystone:SELECT=48,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:59:31.430966 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:00:00.129088 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4271a-52e7-4070-9465-9c8cf02f02d7 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:00.131067 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-2ba4271a-52e7-4070-9465-9c8cf02f02d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:00.142475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4271a-52e7-4070-9465-9c8cf02f02d7 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:00.191838 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1eb7f-c253-41cc-bf2f-304655be3122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:00.192452 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1eb7f-c253-41cc-bf2f-304655be3122 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:00.192731 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1eb7f-c253-41cc-bf2f-304655be3122 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:00.193003 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1eb7f-c253-41cc-bf2f-304655be3122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:00.194795 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-b7d1eb7f-c253-41cc-bf2f-304655be3122 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:00.266049 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1eb7f-c253-41cc-bf2f-304655be3122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:00.267577 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1eb7f-c253-41cc-bf2f-304655be3122 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:00.278560 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1421/2871] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:01.467834 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:01.469318 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:01.483154 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcd7c4-98ea-42f3-8c85-0932345a3384 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:01.486758 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffd42b-a947-4917-90c8-1b92137bdbd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:01.487365 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffd42b-a947-4917-90c8-1b92137bdbd0 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:01.487637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffd42b-a947-4917-90c8-1b92137bdbd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:01.487901 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffd42b-a947-4917-90c8-1b92137bdbd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:01.489594 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6dffd42b-a947-4917-90c8-1b92137bdbd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:01.503716 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffd42b-a947-4917-90c8-1b92137bdbd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:01.505140 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffd42b-a947-4917-90c8-1b92137bdbd0 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:01.516861 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1451/2872] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:03.894090 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3082aa-7f64-480b-89e2-6277fa2cc3c6 None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:03.894688 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3082aa-7f64-480b-89e2-6277fa2cc3c6 None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:03.895071 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3082aa-7f64-480b-89e2-6277fa2cc3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:03.916611 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-ef3082aa-7f64-480b-89e2-6277fa2cc3c6 None None] MFA Rules not processed for user `19c2f15eb1644278b911cc50370da9b7`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:00:03.918607 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-ef3082aa-7f64-480b-89e2-6277fa2cc3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:03.983264 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1422/2873] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 20:00:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:00:11.511280 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=6 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:00:13.975194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=79 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:00:22.338352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffd42b-a947-4917-90c8-1b92137bdbd0 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:22.340114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-6dffd42b-a947-4917-90c8-1b92137bdbd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:22.354909 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffd42b-a947-4917-90c8-1b92137bdbd0 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:22.361492 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a7a27-fb3f-439a-9428-007356b6b50b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:22.362114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a7a27-fb3f-439a-9428-007356b6b50b service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:22.362397 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a7a27-fb3f-439a-9428-007356b6b50b service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:22.362720 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a7a27-fb3f-439a-9428-007356b6b50b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:22.364482 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-985a7a27-fb3f-439a-9428-007356b6b50b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:22.378215 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a7a27-fb3f-439a-9428-007356b6b50b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:22.379832 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a7a27-fb3f-439a-9428-007356b6b50b service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:22.391432 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1452/2874] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:22.456882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3082aa-7f64-480b-89e2-6277fa2cc3c6 None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:22.471947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3082aa-7f64-480b-89e2-6277fa2cc3c6 None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:22.479591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08848ed3-2981-4a69-9d45-184574d58bee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:22.479591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08848ed3-2981-4a69-9d45-184574d58bee service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:22.479591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08848ed3-2981-4a69-9d45-184574d58bee service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:22.479591 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08848ed3-2981-4a69-9d45-184574d58bee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:22.481955 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-08848ed3-2981-4a69-9d45-184574d58bee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:22.498093 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08848ed3-2981-4a69-9d45-184574d58bee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:22.499668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08848ed3-2981-4a69-9d45-184574d58bee service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:22.509951 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1423/2875] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:32.386041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=6 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:00:32.504907 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=6 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:00:36.905441 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a7a27-fb3f-439a-9428-007356b6b50b service nova] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:36.907011 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-985a7a27-fb3f-439a-9428-007356b6b50b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:36.918518 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a7a27-fb3f-439a-9428-007356b6b50b service nova] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:36.922174 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb35f4-de7b-4695-aae1-50ea9d63fe4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:36.922797 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb35f4-de7b-4695-aae1-50ea9d63fe4e service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:36.923075 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb35f4-de7b-4695-aae1-50ea9d63fe4e service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:36.923343 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb35f4-de7b-4695-aae1-50ea9d63fe4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:36.925026 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-d4eb35f4-de7b-4695-aae1-50ea9d63fe4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:36.935087 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb35f4-de7b-4695-aae1-50ea9d63fe4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:36.936739 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb35f4-de7b-4695-aae1-50ea9d63fe4e service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:36.944765 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1453/2876] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:37.071869 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08848ed3-2981-4a69-9d45-184574d58bee service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:37.081723 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08848ed3-2981-4a69-9d45-184574d58bee service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:37.085482 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594acd3-b04b-4321-b1ac-54738c778954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:37.086154 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594acd3-b04b-4321-b1ac-54738c778954 service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:37.086506 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594acd3-b04b-4321-b1ac-54738c778954 service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:37.086871 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594acd3-b04b-4321-b1ac-54738c778954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:37.088741 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-0594acd3-b04b-4321-b1ac-54738c778954 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:37.099473 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594acd3-b04b-4321-b1ac-54738c778954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:37.100939 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594acd3-b04b-4321-b1ac-54738c778954 service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:37.110563 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1424/2877] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:37.185357 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb35f4-de7b-4695-aae1-50ea9d63fe4e service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:37.194590 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb35f4-de7b-4695-aae1-50ea9d63fe4e service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:37.198331 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a9f12-9e54-430b-a0ed-e0b158dc72d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:37.199058 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a9f12-9e54-430b-a0ed-e0b158dc72d7 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:37.199297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a9f12-9e54-430b-a0ed-e0b158dc72d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:37.199694 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a9f12-9e54-430b-a0ed-e0b158dc72d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:37.201673 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-277a9f12-9e54-430b-a0ed-e0b158dc72d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:37.216018 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a9f12-9e54-430b-a0ed-e0b158dc72d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:37.217442 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a9f12-9e54-430b-a0ed-e0b158dc72d7 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:37.228544 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1454/2878] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:37.293789 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594acd3-b04b-4321-b1ac-54738c778954 service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:00:37.295205 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-0594acd3-b04b-4321-b1ac-54738c778954 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:37.368092 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594acd3-b04b-4321-b1ac-54738c778954 service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:00:37.374135 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a94fb-8d74-4121-b8da-fe2bf0eb66db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:00:37.374779 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a94fb-8d74-4121-b8da-fe2bf0eb66db service placement] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:00:37.375053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a94fb-8d74-4121-b8da-fe2bf0eb66db service placement] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:00:37.375321 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a94fb-8d74-4121-b8da-fe2bf0eb66db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:00:37.377055 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-169a94fb-8d74-4121-b8da-fe2bf0eb66db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:00:37.388693 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a94fb-8d74-4121-b8da-fe2bf0eb66db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:00:37.390239 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a94fb-8d74-4121-b8da-fe2bf0eb66db service placement] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:00:37.400016 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1425/2879] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:00:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:00:47.223637 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=12 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:00:47.395880 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=38 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:04.410506 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a9f12-9e54-430b-a0ed-e0b158dc72d7 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:04.412261 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-277a9f12-9e54-430b-a0ed-e0b158dc72d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:04.428409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a9f12-9e54-430b-a0ed-e0b158dc72d7 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:04.439065 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f9206-96b8-4cb7-b776-1a2d71ad092a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:04.440053 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f9206-96b8-4cb7-b776-1a2d71ad092a service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:04.440426 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f9206-96b8-4cb7-b776-1a2d71ad092a service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:04.440789 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f9206-96b8-4cb7-b776-1a2d71ad092a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:04.442851 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-168f9206-96b8-4cb7-b776-1a2d71ad092a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:04.457413 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f9206-96b8-4cb7-b776-1a2d71ad092a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:04.459055 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f9206-96b8-4cb7-b776-1a2d71ad092a service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:04.470315 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1455/2880] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:09.342178 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991ed18-80e8-480d-801f-a3998c7a574d None None] REQUEST_METHOD: `POST` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:09.342784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991ed18-80e8-480d-801f-a3998c7a574d None None] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:09.342784 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991ed18-80e8-480d-801f-a3998c7a574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:09.363540 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.auth.core [None req-7991ed18-80e8-480d-801f-a3998c7a574d None None] MFA Rules not processed for user `19c2f15eb1644278b911cc50370da9b7`. Rule list: `[]` (Enabled: `True`). {{(pid=87005) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 20:02:09.364788 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7991ed18-80e8-480d-801f-a3998c7a574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:09.371763 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1426/2881] 10.222.0.86 () {58 vars in 958 bytes} [Fri Aug 5 20:02:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 05 20:02:09.689895 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f9206-96b8-4cb7-b776-1a2d71ad092a service nova] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:09.699231 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f9206-96b8-4cb7-b776-1a2d71ad092a service nova] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:09.702812 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e3b13-ef98-4140-b911-d98baa14fb53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:09.703414 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e3b13-ef98-4140-b911-d98baa14fb53 service neutron] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:09.703708 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e3b13-ef98-4140-b911-d98baa14fb53 service neutron] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:09.704519 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e3b13-ef98-4140-b911-d98baa14fb53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:09.706048 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4b5e3b13-ef98-4140-b911-d98baa14fb53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:09.717771 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e3b13-ef98-4140-b911-d98baa14fb53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:09.719265 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e3b13-ef98-4140-b911-d98baa14fb53 service neutron] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:09.729817 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1456/2882] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:14.198450 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991ed18-80e8-480d-801f-a3998c7a574d None None] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:14.199971 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7991ed18-80e8-480d-801f-a3998c7a574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:14.216981 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991ed18-80e8-480d-801f-a3998c7a574d None None] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:14.223053 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ecd26-f457-49ee-b4d0-b3fc3997149e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:14.224035 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ecd26-f457-49ee-b4d0-b3fc3997149e service placement] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:14.224576 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ecd26-f457-49ee-b4d0-b3fc3997149e service placement] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:14.225217 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ecd26-f457-49ee-b4d0-b3fc3997149e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:14.227407 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d62ecd26-f457-49ee-b4d0-b3fc3997149e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:14.242116 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ecd26-f457-49ee-b4d0-b3fc3997149e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:14.244146 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ecd26-f457-49ee-b4d0-b3fc3997149e service placement] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:14.262376 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1427/2883] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:18.265253 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e3b13-ef98-4140-b911-d98baa14fb53 service neutron] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:18.267474 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4b5e3b13-ef98-4140-b911-d98baa14fb53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:18.346232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e3b13-ef98-4140-b911-d98baa14fb53 service neutron] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:18.353929 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:18.354588 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:18.354911 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:18.355191 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] PATH_INFO: `/v3/users/802ed5c51d184f6c925f31ab6020ac93` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:18.356484 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=802ed5c51d184f6c925f31ab6020ac93)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:18.358409 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:18.410167 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '802ed5c51d184f6c925f31ab6020ac93'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.424913 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '802ed5c51d184f6c925f31ab6020ac93'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.432362 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '802ed5c51d184f6c925f31ab6020ac93'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.433102 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '802ed5c51d184f6c925f31ab6020ac93'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.433721 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '802ed5c51d184f6c925f31ab6020ac93'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.446905 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1457/2884] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:18 2022] DELETE /identity/v3/users/802ed5c51d184f6c925f31ab6020ac93 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:18.627359 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ecd26-f457-49ee-b4d0-b3fc3997149e service placement] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:18.639980 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ecd26-f457-49ee-b4d0-b3fc3997149e service placement] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:18.698989 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:18.699511 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:18.699785 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:18.700052 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] PATH_INFO: `/v3/projects/47ea1b6ed7d647b8987e6c128248cab6` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:18.701196 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ea1b6ed7d647b8987e6c128248cab6)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:18.703113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:18.749604 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ea1b6ed7d647b8987e6c128248cab6', 'request_id': 'req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.756320 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ea1b6ed7d647b8987e6c128248cab6', 'request_id': 'req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.763730 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ea1b6ed7d647b8987e6c128248cab6', 'request_id': 'req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.774808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] The token cache is being invalidate because project 47ea1b6ed7d647b8987e6c128248cab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:02:18.775091 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.775739 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.777266 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1428/2885] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 20:02:18 2022] DELETE /identity/v3/projects/47ea1b6ed7d647b8987e6c128248cab6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 20:02:18.781300 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:18.782855 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:18.795717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6e028-221f-4fc1-b861-af12421ae113 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:18.843037 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:18.853159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:18.853788 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:18.854066 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:18.854334 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] PATH_INFO: `/v3/users/6c2457e56a1841c0907777cbf6284921` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:18.855532 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c2457e56a1841c0907777cbf6284921)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:18.857435 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:18.902295 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c2457e56a1841c0907777cbf6284921'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.914798 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c2457e56a1841c0907777cbf6284921'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.922032 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c2457e56a1841c0907777cbf6284921'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.923716 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c2457e56a1841c0907777cbf6284921'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.924324 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c2457e56a1841c0907777cbf6284921'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:18.936232 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1458/2886] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 20:02:18 2022] DELETE /identity/v3/users/6c2457e56a1841c0907777cbf6284921 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:19.141799 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:19.143468 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:19.156193 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e29d8-9f0e-46d5-95d1-adb7a23ad488 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:19.225621 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:19.226259 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:19.226540 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:19.226844 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] PATH_INFO: `/v3/projects/a05914c4ddfa4c72b485a28548b9d83e` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:19.228002 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a05914c4ddfa4c72b485a28548b9d83e)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:19.229882 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:19.241988 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:19.280918 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a05914c4ddfa4c72b485a28548b9d83e', 'request_id': 'req-d034cf67-4616-4239-836f-cb50fbfbbf39'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:19.288009 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a05914c4ddfa4c72b485a28548b9d83e', 'request_id': 'req-d034cf67-4616-4239-836f-cb50fbfbbf39'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:19.296240 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a05914c4ddfa4c72b485a28548b9d83e', 'request_id': 'req-d034cf67-4616-4239-836f-cb50fbfbbf39'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:19.307163 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] The token cache is being invalidate because project a05914c4ddfa4c72b485a28548b9d83e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87005) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:02:19.307452 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:19.308110 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:02:19.309683 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1429/2887] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 20:02:19 2022] DELETE /identity/v3/projects/a05914c4ddfa4c72b485a28548b9d83e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:02:20.255292 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:20.256767 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:20.330449 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacef933-228a-46db-9943-42a90f99cfc1 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:20.393742 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d86f4-843b-4439-a950-743d77320471 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:20.394355 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d86f4-843b-4439-a950-743d77320471 service nova] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:20.394635 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d86f4-843b-4439-a950-743d77320471 service nova] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:20.394927 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d86f4-843b-4439-a950-743d77320471 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:20.396659 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7b7d86f4-843b-4439-a950-743d77320471 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:20.402567 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:20.459008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d86f4-843b-4439-a950-743d77320471 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:20.460717 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d86f4-843b-4439-a950-743d77320471 service nova] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:20.470633 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1459/2888] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 05 20:02:24.066011 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:24.067684 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:24.085589 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034cf67-4616-4239-836f-cb50fbfbbf39 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:24.091201 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704f3f0-0eb6-42ec-93cf-6d1d40226440 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab8f9d9cf4ca4e38ae6eb67e949b40fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:24.091963 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704f3f0-0eb6-42ec-93cf-6d1d40226440 service nova] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:24.092142 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704f3f0-0eb6-42ec-93cf-6d1d40226440 service nova] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:24.092362 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704f3f0-0eb6-42ec-93cf-6d1d40226440 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:24.094140 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e704f3f0-0eb6-42ec-93cf-6d1d40226440 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:24.107740 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704f3f0-0eb6-42ec-93cf-6d1d40226440 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:24.109383 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e704f3f0-0eb6-42ec-93cf-6d1d40226440 service nova] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:24.121760 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1430/2889] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:02:30.465586 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=24 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:34.116475 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:02:55.601718 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d86f4-843b-4439-a950-743d77320471 service nova] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:02:55.603124 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-7b7d86f4-843b-4439-a950-743d77320471 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:55.669620 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d86f4-843b-4439-a950-743d77320471 service nova] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:02:55.674737 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c75d1e-d4ff-4c18-9399-78d862065332 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:02:55.675344 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c75d1e-d4ff-4c18-9399-78d862065332 service placement] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:02:55.675615 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c75d1e-d4ff-4c18-9399-78d862065332 service placement] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:02:55.675877 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c75d1e-d4ff-4c18-9399-78d862065332 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:02:55.677593 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-b5c75d1e-d4ff-4c18-9399-78d862065332 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:02:55.689352 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c75d1e-d4ff-4c18-9399-78d862065332 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:02:55.691207 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c75d1e-d4ff-4c18-9399-78d862065332 service placement] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:02:55.701071 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1460/2890] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:02:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:03:05.696809 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=32 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:10.250990 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704f3f0-0eb6-42ec-93cf-6d1d40226440 service nova] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:10.252454 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e704f3f0-0eb6-42ec-93cf-6d1d40226440 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:10.310089 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704f3f0-0eb6-42ec-93cf-6d1d40226440 service nova] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:10.313689 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0411ab4-d7c2-4324-b749-e0f5a3de3f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80097afc3ce540e6a8060f696535e62c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:10.314294 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0411ab4-d7c2-4324-b749-e0f5a3de3f0d service neutron] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:10.314571 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0411ab4-d7c2-4324-b749-e0f5a3de3f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:10.314873 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0411ab4-d7c2-4324-b749-e0f5a3de3f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:10.316975 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e0411ab4-d7c2-4324-b749-e0f5a3de3f0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:10.380204 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0411ab4-d7c2-4324-b749-e0f5a3de3f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:10.381641 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0411ab4-d7c2-4324-b749-e0f5a3de3f0d service neutron] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:10.391245 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1431/2891] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:04:16.442715 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c75d1e-d4ff-4c18-9399-78d862065332 service placement] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:16.454837 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c75d1e-d4ff-4c18-9399-78d862065332 service placement] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:16.463291 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c36bc-c3c4-43b1-bc83-78f185dd0e67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:16.464178 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c36bc-c3c4-43b1-bc83-78f185dd0e67 service placement] REQUEST_METHOD: `GET` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:16.464696 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c36bc-c3c4-43b1-bc83-78f185dd0e67 service placement] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:16.465130 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c36bc-c3c4-43b1-bc83-78f185dd0e67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:16.467229 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-4f6c36bc-c3c4-43b1-bc83-78f185dd0e67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:16.506041 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c36bc-c3c4-43b1-bc83-78f185dd0e67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:16.508336 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c36bc-c3c4-43b1-bc83-78f185dd0e67 service placement] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:16.529008 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1461/2892] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:04:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:04:16.540384 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0411ab4-d7c2-4324-b749-e0f5a3de3f0d service neutron] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:16.541829 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-e0411ab4-d7c2-4324-b749-e0f5a3de3f0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:16.556561 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0411ab4-d7c2-4324-b749-e0f5a3de3f0d service neutron] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:16.564786 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:16.565947 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:16.566438 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:16.567206 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] PATH_INFO: `/v3/users/19c2f15eb1644278b911cc50370da9b7` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:16.568793 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c2f15eb1644278b911cc50370da9b7)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:16.571194 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:16.638668 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c2f15eb1644278b911cc50370da9b7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:16.647326 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c2f15eb1644278b911cc50370da9b7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:16.648279 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c2f15eb1644278b911cc50370da9b7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:16.669236 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c2f15eb1644278b911cc50370da9b7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:16.681638 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c2f15eb1644278b911cc50370da9b7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:16.688276 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1432/2893] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 20:04:16 2022] DELETE /identity/v3/users/19c2f15eb1644278b911cc50370da9b7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:16.875691 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c36bc-c3c4-43b1-bc83-78f185dd0e67 service placement] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:16.887114 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c36bc-c3c4-43b1-bc83-78f185dd0e67 service placement] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:16.944183 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:16.944819 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:16.945100 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:16.945373 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] PATH_INFO: `/v3/projects/7372aebdf6554202a45b0bbd7bad0700` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:16.946530 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7372aebdf6554202a45b0bbd7bad0700)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:16.948454 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:16.993757 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7372aebdf6554202a45b0bbd7bad0700', 'request_id': 'req-fa6c5a6b-5473-4690-bc79-65a75f8135a8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:16.999941 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7372aebdf6554202a45b0bbd7bad0700', 'request_id': 'req-fa6c5a6b-5473-4690-bc79-65a75f8135a8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.007826 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7372aebdf6554202a45b0bbd7bad0700', 'request_id': 'req-fa6c5a6b-5473-4690-bc79-65a75f8135a8'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.019119 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] The token cache is being invalidate because project 7372aebdf6554202a45b0bbd7bad0700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:04:17.019380 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.020095 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.021649 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1462/2894] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 20:04:16 2022] DELETE /identity/v3/projects/7372aebdf6554202a45b0bbd7bad0700 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:17.025630 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:17.027171 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:17.041308 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db6826-cb5c-4f4f-a5e9-0f16c249b3d2 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:17.059931 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:17.103871 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:17.104554 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:17.104915 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:17.105190 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] PATH_INFO: `/v3/users/d60f1886601b4e5d911146fb371f28b7` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:17.106469 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d60f1886601b4e5d911146fb371f28b7)` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:17.108432 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:17.162970 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd60f1886601b4e5d911146fb371f28b7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.170903 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd60f1886601b4e5d911146fb371f28b7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.171609 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd60f1886601b4e5d911146fb371f28b7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.184647 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd60f1886601b4e5d911146fb371f28b7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.194585 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.notifications [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd60f1886601b4e5d911146fb371f28b7'} {{(pid=87005) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.200751 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1433/2895] 10.222.0.86 () {58 vars in 1224 bytes} [Fri Aug 5 20:04:17 2022] DELETE /identity/v3/users/d60f1886601b4e5d911146fb371f28b7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:17.380416 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] Authenticating user token {{(pid=87004) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:04:17.381865 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.fernet_utils [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87004) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:04:17.395894 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c5a6b-5473-4690-bc79-65a75f8135a8 admin admin] Validating token access rules against request {{(pid=87004) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:04:17.454222 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88c5b6-12af-42a2-9760-0a174d3e245c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '047b4b41672048d8889a2b3bb55b2d97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11aa9c879322427d8920955a2aa93539', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87004) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:04:17.454946 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88c5b6-12af-42a2-9760-0a174d3e245c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:04:17.455159 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88c5b6-12af-42a2-9760-0a174d3e245c admin admin] SCRIPT_NAME: `/identity` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:04:17.455460 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88c5b6-12af-42a2-9760-0a174d3e245c admin admin] PATH_INFO: `/v3/projects/841a6466aef743a8b19c027d3c964e7b` {{(pid=87004) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:04:17.457297 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88c5b6-12af-42a2-9760-0a174d3e245c admin admin] RBAC: Authorizing `identity:delete_project(project_id=841a6466aef743a8b19c027d3c964e7b)` {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:04:17.459243 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88c5b6-12af-42a2-9760-0a174d3e245c admin admin] RBAC: Authorization granted {{(pid=87004) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:04:17.480603 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:17.508764 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2e88c5b6-12af-42a2-9760-0a174d3e245c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '841a6466aef743a8b19c027d3c964e7b', 'request_id': 'req-2e88c5b6-12af-42a2-9760-0a174d3e245c'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.514688 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2e88c5b6-12af-42a2-9760-0a174d3e245c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '841a6466aef743a8b19c027d3c964e7b', 'request_id': 'req-2e88c5b6-12af-42a2-9760-0a174d3e245c'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.522528 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2e88c5b6-12af-42a2-9760-0a174d3e245c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '841a6466aef743a8b19c027d3c964e7b', 'request_id': 'req-2e88c5b6-12af-42a2-9760-0a174d3e245c'} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.533635 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2e88c5b6-12af-42a2-9760-0a174d3e245c admin admin] The token cache is being invalidate because project 841a6466aef743a8b19c027d3c964e7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87004) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 05 20:04:17.533993 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2e88c5b6-12af-42a2-9760-0a174d3e245c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.534638 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG keystone.notifications [None req-2e88c5b6-12af-42a2-9760-0a174d3e245c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87004) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 05 20:04:17.536181 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: [pid: 87004|app: 0|req: 1463/2896] 10.222.0.86 () {58 vars in 1233 bytes} [Fri Aug 5 20:04:17 2022] DELETE /identity/v3/projects/841a6466aef743a8b19c027d3c964e7b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 20:04:27.197211 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:04:27.530583 dv-cinder-852209-2-iscsi devstack@keystone.service[87004]: DEBUG dbcounter [-] [87004] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87004) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 20:08:01.519284 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] Authenticating user token {{(pid=87005) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 20:08:01.520808 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:01.662923 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508253e-c718-4cb8-9145-7f9fa2a52ee2 admin admin] Validating token access rules against request {{(pid=87005) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 20:08:01.720409 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161152f-e687-4a23-bfbf-94762a64dbe9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce71f93c80d433b830c448d0a7a14e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c73d2cfaa1c47829ca6936186d2bb26', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87005) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 20:08:01.721075 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161152f-e687-4a23-bfbf-94762a64dbe9 service placement] REQUEST_METHOD: `GET` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 20:08:01.721377 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161152f-e687-4a23-bfbf-94762a64dbe9 service placement] SCRIPT_NAME: `/identity` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 20:08:01.721682 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161152f-e687-4a23-bfbf-94762a64dbe9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87005) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 20:08:01.723470 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.fernet_utils [None req-1161152f-e687-4a23-bfbf-94762a64dbe9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87005) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 20:08:01.735933 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161152f-e687-4a23-bfbf-94762a64dbe9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 20:08:01.737429 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161152f-e687-4a23-bfbf-94762a64dbe9 service placement] RBAC: Authorization granted {{(pid=87005) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 20:08:01.748527 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: [pid: 87005|app: 0|req: 1434/2897] 10.222.0.86 () {60 vars in 1365 bytes} [Fri Aug 5 20:08:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 05 20:08:11.744113 dv-cinder-852209-2-iscsi devstack@keystone.service[87005]: DEBUG dbcounter [-] [87005] Writing DB stats keystone:SELECT=89 {{(pid=87005) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}